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1.0  INTRODUCTION 


1.1  Background 

In  ihe  early  197()’s  the  United  Stales  Air  Force  recognized  a  need  to  establish  a  center  to  serve  as 
a  focal  point  for  software  engineering  and  software  technology  information  and  software 
experience  data.  The  center  would  ideally  serve  the  combined  technical  needs  of  the 
government,  industry,  and  the  academic  communities.  In  1976,  The  Rome  Air  Development 
Center  (RADC),  now  Rome  Laboratory  (RL)  contracted  the  design  of  a  center  tasked  to  acquire, 
store,  analyze,  synthesize,  and  report  software  engineering/iechnology  information.  In  1978,  RL 
contracted  the  development  of  the  center,  to  be  named  the  Data  &  Analysis  Center  for  Software 
(DACS). 

In  January  1981,  the  DACS  achieved  status  as  a  Department  of  Defcn.se  Information  Analysis 
Cent^T  (DoD  lAC),  while  still  in  its  test  period.  A  primary  goal  of  the  change  in  status  was  to 
transition  the  DACS  irom  a  U.S.  Air  Force  Rome  Air  Development  Center  (now  Rome 
Laboratory  program  to  a  DoD  information  resource. 

The  originating  contractor,  IIT  Research  Institute,  operated  the  DACS  until  the  award  of  this 
three  year  contract,  F30602-89-C-(X)82,  in  1989.  This  report  describes  the  activities  and  actions 
of  the  current  DACS  contractor  during  the  period  of  performance  of  this  contract,  1989  -  1992. 


1.2  Objectives 

Primarily,  the  DACS  was  established  with  the  goal  to  serve  as  a  focal  point  for  the  defense 
software  engineering  community  for  issues  involving  .software  development  and  the  collection  of 
experience  data  relating  to  .software  engineering  efforts  within  the  DoD.  To  achieve  the  goal, 
the  DACS  statement  of  work  requires  the  DACS  contractor  to  provide  .software  engineering 
expertise  to  the  defense  community;  collect,  analyze,  synthesize  and  disseminate  software 
engineering  data;  collect  and  report  scientific  and  technical  information  (STINFO)  within  the 
field;  and  provide  the  community  with  a  centralized  authoritative  source  for  current  and  readily 
available  .software  engineering  information. 

The  DACS  is  organized  to  meet  the  objectives  outlined  above  through  the  following: 

•  Providing  products  and  .services  to  the  defen.se  community  to  aid  in  the  .software 
engineering  technology  transition  and  transfer. 

•  Support  for  software  technology  transfer  through  conferences  and  workshops  aimed  at 
dis.semination  and  exchange  of  existing  information  and  developmental  technology. 

•  Making  software  engineering  project,  tool,  and  life  cycle  data  available  to  DACS  u.sers. 

•  Analyzing  software  engineering  data  and  providing  that  analy.sis  to  the  reque.sters  and  to 
other  users  as  appropriate. 


Providing  information  transfer  to  our  u.sers  through  publications  such  as  the  DACS 
Newsletter  and  periodic  bulletin.s. 


•  Preparing  technology  reports  describing  the  state-ot’-the-  art  and  practice  of  selected 
software  technology  areas. 

•  Preparing  critical  reviews  and  technology  assessments  to  take  a  more  in-depth  look  at 
selected  software  engineering  technology  areas. 

•  Conducted  re.search  into  various  aspects  of  software  engineering  to  improve  methods, 
performance,  .suitability,  quality,  and  reliability  of  software. 

•  Minimizing  duplication  of  software  research  and  engineering  through  program  review 
and  coordination  efforts. 

•  Providing  technical  and  engineering  a.ssi.stance  to  DACS  u.sers  for  the  solution  of 
questions  and  problems  concerning  .software  engineering/technology  issues. 

To  meet  the  general  goais  and  objectives  of  the  DACS,  the  Kaman  Sciences  Corporation  team 
developed  .several  goals  which  it  pursued  during  the  performance  of  this  contract.  They  are  as 
follows: 

•  Increa.se  the  visibility  of  the  DACS  as  a  .software  engineering  resource  available  to  the 
defense  community. 

•  Enhance  the  STINFO  program  holdings. 

•  Improve  the  type  and  level  of  products  and  .services  available  to  the  u.sens. 

•  Provide  cost  effective  operation  of  the  DACS  in  an  era  of  research  and  development 
expansion  but  with  the  limited  re.sources  available  for  DACS  operation. 

In  the  remaining  .sections  of  this  report,  we  will  highlight  the  various  aspects  of  the  DACS 
Charter  and  the  succe.s.ses  and  lessons  learned  through  Kaman’s  initial  contract  as  the  DACS 
operator. 


1.3  Contents 

This  report  is  divided  into  1 1  .sections.  The  following  lists  each  .section  and  provides  a  brief 
description  of  the  contents: 

Section  1.0  Background,  goals  and  objectives  of  the  DACS. 

Section  2.0  General  operations  of  the  DACS. 

Section  3.0  Dc.scription  and  discu.ssion  of  the  .scientific  and  technical  information  program 
Section  4.0  Di.scu.ssion  of  the  data  acqui.sition  efforts  for  the  period. 

Section  5.0  Dc.scription  of  the  data  analy.sis  program  conducted  during  this  contract. 
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Section  6.0  Technical  rcports  produced  during  this  conyaci  period. 

Section  7.0  Summary  of  the  information  transfer  efforts  within  the  DACS. 

Seeiion  8.0  Description  of  new  produces  and  services  available  from  the  DACS. 

Section  9.0  Promotional  efforts  made  to  increase  DACS  visibility. 

Section  10.0  Technical  area  task  summaries. 

Section  1 1.0  Lessons  learned  and  recommendations  for  the  future. 


2.0  DACS  OPERATIONS  AND  MAINTENANCE 

Throughout  the  performance  of  this  contract.  Kaman  Sciences  Corporation  sought  to  apply  the 
highest  level  of  software  engineering  talent  available  to  solve  problems  brought  forward  in  the 
software  engineering  community  and  to  provide  top-level  products  and  .services  to  our  u.sers. 
The  DACS  staff,  enhanced  by  the  presence  and  availability  of  numerous  Kaman  employees  and  a 
strong  subcontracting  team,  provided  thorough  coverage  of  the  .software  engineering  disciplines 
and  effective  solutions  to  the  needs  of  the  defense  .software  engineering  community. 

Efforts  were  expended  to  increase  the  quality  and  quantity  of  our  .services,  enhance  the  level  and 
type  of  products  and  services  we  provided  and  increase  the  visibility  of  the  DACS  within  all 
areas  of  the  .software  engineering  community.  To  accomplish  our  plans  to  upgrade  the 
capabilities  and  reputation  of  the  DACS  we  pursued  a  number  of  activities. 

Technical  reports  in  the  form  of  state-of-the-art  and  critical  review/technology  a.s.sessments  were 
produced  on  topics  of  major  interesi  to  members  of  the  defense  community.  (See  section  8.0  for 
a  di.scussion  of  the  technical  report  products.)  While  many  of  the  reports  were  prepared  by 
members  of  the  DACS  staff,  other  reports  were  prepared  on  a  sub-contracted  basis,  with  industry 
and  academic  leaders  in  the  software  Held. 

We  added  substantial  numbers  of  .software  engineering  abstracts  and  citations  from  software 
engineering  and  software  technology  papers,  proceedings,  books  and  other  printed  source 
material.  Approved  citations  increased  from  over  six  thou.sand  approved  citations  produced  in 
the  first  ten  years  of  DACS  existence  to  more  than  thirteen  thou,sand  citations  at  the  end  of  this 
contract.  We  added  citations  from  newly  available  material  as  well  as  filled  in  missing  citations 
of  important  material  from  the  mid-198()s  to  the  present.  We  also  reversed  the  trend  in  adding 
citations  without  having  the  corresponding  source  material  also  available.  Important 
publications,  books  and  proceedings  as  well  as  other  supporting  printer  matter  were  again  added 
to  the  DACS  holdings  as  well. 

The  DACS  staff  provided  expert  scientific  and  engineering  assistance  in  solving  a  wide  variety 
of  special  studies  for  our  u.sers.  We  were  able  to  .serve  not  only  the  traditional  customers  of  the 
DACS  but  were  able  to  attract  a  number  of  new  customers  as  well. 

Upgrades  to  the  DACS  New.sletter,  published  quarterly,  proved  very  popular  with  the  DACS  user 
community.  In  a  survey  of  our  .sub.scribers.  one  of  the  most  frequent  comments  was  that  the 


improved  format,  increased  readability  of  technical  articles  and  general  content  was  a  positive 
incentive  for  customer  use  of  the  DACS  and  its  products  and  services. 


OACS  USER  OtSlRIBimON 


FIGURE  2,0  DACS  USER  DISTRIBUTION 
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To  increase  the  visibility,  recognition  and  reputation  of  the  DACS  we  increased  our  involvement 
with  high-level  activities  whose  goals  included  long  term  software  engineering  planning  and 
specification.  The  DACS  staff  actively  supported  efforts  involved  with  the  following; 

•  DoD  Software  Action  Plan 

•  DoD  Software  Technology  Strategy 

•  Standards  and  Specifications  Identification.  Selection,  and  Recommendation 

•  Militarily  Critical  Technologies  List  Planning 

•  Navy  Next  Generation  Computer  Resources  Program 

•  National  Organization  Support  (e.g.,  IEEE.  ACM.  AIAA,  etc.) 

Through  our  efforts  we  directly  supported  the  Defen.se  Science  and  Technok.gy  Strategy  for 
maintaining  military  technology  superiority.  We  participated  in  key  re.search  and  development 
areas  that  included  artificial  neural  networks,  distributed  and  parallel  processing  in 
heterogeneous  and  homogeneous  environment.s.  signal  priKcssing  algorithms,  software  image 
proce.ssing,  risk  assessment,  the  development  and  u.se  of  artificial  intelligence  tools  and 
techniques,  and  .software  producibility  measures. 

The  DACS  also  provided  support  for  De.sert  Storm  and  De.sert  Shield  through  performance  of 
.special  studies.  The  DACS  has  conducted  a  .series  of  special  studies  for  the  U.S.  Army 
Armament,  Munitions,  and  Chemical  Command  (AMCCOM)  investigating  the  u.se  of 
application  programming  environments  within  the  context  of  open  systems  architecture.  The 
result  of  the.se  studies  has  been  the  development  of  a  prototype  advanced  technology  office 
automation  system  which  u.ses  a  commercial  o'.-the-.sheIf  programming  system  in  conjunction 
with  relational  databa.ses  to  provide  a  highly  portable,  easy  to  maintain,  u.ser  friendly  tool  for  the 
Army.  It  is  planned  during  the  next  DACS  contract  to  make  the  tools  developed  by  Kaman 
personnel  available  as  products  through  the  DACS. 

The  prototype  Technical  Data  Package  Tracking  and  Reporting  System,  known  as  the  TDP 
TRACKER,  automates  the  generation  and  tran.smi.ssion  of  the  technical  data  associated  with  a 
procurement  package  within  the  U.S.  Army  Armament,  Munitions,  and  Chemical  Command 
(AMCCOM).  The  TRACKER  handles  the  automatic  consolidation  and  authorization  of  the 
Contract  Data  Requirements  List  (CDRL),  Document  Summary  List  (DSL),  the  Statement  of 
Work  (SOW),  and  the  on-line  query  of  exi.sting  PrtKurement  Work  Directive  (PWD)  data.  The 
system  is  capable  of  routing  text  and  image  data  and  has  significantly  reduced  the  average 
response  time  (from  180  days  to  less  than  40)  and  improved  the  quality  of  technical  data 
packages. 

Operating  as  a  distributed  processing  system,  the  TRACKER  receives  input  daily  from 
AMCCOM's  Commodity  Command  Supply  Sy.stem  in  the  form  of  PWDs  for  which  technical 
data  packages  may  need  to  be  generated.  TRACKER  parses  the  PWD  images  for  key  words  and 
phrases,  and,  using  rules,  distributes  the.se  images  to  those  organizations  from  which 
inpul/approval  must  be  obtained.  The.se  organizations  are  located  at  the  Rock  Island  Arsenal, 
Rock  Island,  Illinois  and  at  the  Picatinny  Arsenal,  Dover,  New  Jersey.  Chemical  actions  are 
queued  for  the  Chemical  Re.search  and  Development  Center,  Aberdeen,  Md.,  but  the  chemical 
function  is  not  included  in  the  prototype. 


During  the  Desert  ShieltiySlorm  operation.  TRACKER  handled  die  ron  elassilied  proeuienie.u 
actions  tor  AMCCOM.  Numbering  approximately  d(KK).  these  actions  were  given  special  priority 
cooes  which  grouped  them  for  immediate  action  within  each  organi/ation's  work  queues,  t'sing 
the  TOACKER's  distributed  query  processing,  organizations  could  preview  and  prepare  bir  items 
that  had  not  yet  reached  their  organi/atitms.  and  could  perl’orm  their  required  ..ctions  as  soon  as 
the  system  routed  items  to  them. 

With  the  close  of  Desert  .Storm,  the  TRACKER  provides  an  easy  mechanism  lor  review  and 
cancel  of  items  no  longer  required,  and  as  a  repository  of  information  tha’  ciuild  he  used  for  a 
performance  analysis  of  organizations  in  the  technical  data  processing  loop. 
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3.0  DACS  STINFO  PROGRAM 


Scientilic  and  Technical  Information  (STINFO)  may  include  a  variety  of  information  categories. 
At  the  DACS,  we  include  in  STINFO,  the  abstracts  and  citations  that  comprise  our  Software 
Engineering  Bibliographic  Database  (SEBD),  information  and  data  from  software  engineering 
research  which  we  house  in  the  DACS  Software  Engineering  Research  Projects  (SERP) 
Database,  and  information  on  tools  and  technology  developed  to  solve  software  engineering  ind 
software  technology  problems.  That  information  is  maintained  in  the  DACS  Software 
Engineering  Tools  Information  (SETI)  Database.  While  the  latter  databa.  e  is  not  a  contractual 
obligation,  we  feel  that  it  is  important  to  maintain  the  information  for  the  convenience  of  our 
users. 


3.1  Software  Engineering  Bibliographic  Database  (SEBD) 

The  SEBD  has  been  organized,  developed  and  continuously  upgraded  to  insure  a  readily 
acce.ssible  source  of  comprehensive,  current  information  is  available  to  the  DACS  staff  and  our 
U'^ers  on  virtually  all  aspects  of  software  engineering  and  software  technology.  The  information 
is  routinely  acces.sed  in  the  performance  of  DACS  CORE  and  Special  Study  activities. 
Periodically,  the  infonnaiion  is  acce.sscd  by  members  of  our  user  community  as  well. 

Users  may  access  the  data  by  obtaining  printed  copies  of  our  annotated  bibliographies  or  through 
special  reports  detailing  limited  .sets  of  information  ba.sed  on  key-word  identification.  Future 
plans  include  making  the  SEBD  available  via  floppy  disk  in  both  a  PC  version  and  a  Macintosh 
version  and  making  the  SEBD  fully  available  to  our  u.sers  via  on-line  computer  access. 

The  DACS  maintains  more  than  l.^,(KK)  citations  in  the  SEBD.  At  the  clo.se  of  the  contract 
13,  608  citations  were  in  the  dalaba.se  with  1,190  .software  engineering  key  words  contained  in 
the  SEBD  Thesaurus.  Data  was  acquired  from  a  variety  of  texts,  technical  reports,  profe.ssional 
journal  articles,  conference  and  .seminar  proceedings,  academic  thesis  and  papers,  and  other 
pertinent  printed  matter.  Entries  were  made  by  university  student  co-op  students  under  the 
direction  ol  the  DACS  Librarian  and  STINFO  Manager  and  by  students  at  Arizona  State 
University,  a  university  noted  for  an  excellent  .software  engineering  academic  library  collection. 
There  are  a  number  of  ways  to  re.scarch  and  access  the  information  contained  in  the  SEBD.  Most 
typically  data  .searches  arc  carried  out  using  key  word  .searches  from  the  DACS  The.saurus. 


SOFTWARE  ENGINEERING  BIBLIOGRAPHY 
CITATIONS  6  DOCUMENTS _ 
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FIGURE  3.1  SOFTWARE  ENGINEERING  BIBLIOGRAPHIC  DATABASE  GROWTH 


The  staff  of  computer  scientists  and  engineers  at  the  DACS  constantly  seek  to  improve  the  quality 
and  accessibility  of  the  SEBD  data.  Through  enhancement  of  the  input  proces.ses,  data  loading  has 
been  improved  and  the  capability  to  automate  error  checking  schemas  was  under  constant  review, 
A  by-product  of  these  efforts  was  that  the  time  spent  in  performing  clerical  tasks  was  reduced 
while  the  quality  and  accessibility  of  the  SEBD  was  improved.  This  area  was  one  of  ongoing 
commitment  for  KSC. 


3.2  Software  Engineering  Research  Projects  (SERF)  Database 

The  Software  Engineering  Research  Projects  (SERF)  Database  contains  information  on  past  and 
current  research  projects  which  involve  software  engineering  and  software  technology.  Included 
in  the  database  is  a  de.scription  of  the  research,  identification  of  the  re.searcher(s),  the  sponsor  of 
the  particular  effort,  and  available  information  in  the  following  categories 

•  Goals  and  Objectives  •  Level  of  Effort 

•  Technical  Approach  •  Status 

•  Task  Plans  •  Re.sults 

Current  data  in  the  SERF  captures  re.search  project  data  on  a  variety  of  software  engineering 
projects  involving  .software  programming  languages,  models  and  methods,  software  tool 
development  and  application,  as  well  as  information  relating  to  modem  software  programming 
practices. 

The  SERF  is  currently  implemented  as  a  collection  of  hard  copy  data,  a  related  database  acquired 
during  this  contract,  and  third-party  data  accc.ssible  by  the  DACS.  Hard  copy  reports  include 
1,4(X)  Work  Unit  Summaries  acquired  from  the  Defen.se  Technical  Information  Center  (DTIC)  and 
250  citations  from  USAF  Manufacturing  Technology.  DoD  project  monitors  submit  Work  Unit 
Summaries  to  DTIC  to  prevent  duplication  of  effort  on  DoD  efforts.  They  contain  project 
information  such  as  period  of  performance,  points  of  con'act,  funding  level,  keywords  and 
descriptors,  and  a  brief  verbal  summary.  The  DACS  regularly  obtains  hard  copy  Work  Unit 
Summaries  from  DTIC  .selected  on  keywords  related  to  software  engineering  and  software 
technology.  DTIC  is  planning  to  soon  provide  Work  Unit  Summaries  on  CD-ROM. 

The  DACS  has  acquired  the  Robotics  and  Artificial  Intelligence  Databa.se  (RAID)  during  the  cuirent 
contract.  RAID  contains  re.search  project  information  on  robotics  and  Al-related  projects.  It  is 
currently  implemented  as  an  on-line  databa.se  at  the  DACS.  RAID  development  was  funded  by  the 
Army,  the  Marine  Corps,  the  Navy,  and  the  Defen.se  Advanced  Re.search  Frojects  Agency 
(DARFA),  with  the  Naval  CX'ean  Sy.stems  Center  (NOSC)  acting  as  project  monitor.  It  is  de.signed 
as  a  decision  support  .sy.stem  for  program  managers  and  as  a  technical  knowledge  ba.se  for  the 
researcher  in  Robotics  and  Al.  The  project  and  contact  data  stored  in  RAID  is  gathered  from  and 
verified  by  a  variety  of  .sources,  including  DTIC,  the  DoD  Manufacturing  Technology  Frogram, 
and  researchers  in  the  field.  RAID  now  contains  data  on  over  2,000  research  projects  and  3,0(K) 
points  of  contact.  Government-spon.sored  DACS  u.scrs  can  order  reports  from  RAID,  while 
government  personnel  can  have  direct  on-line  acce.ss  over  Internet.  On-line  access  permits  the  u.ser 
to  search  RAID  directly  and  print  the  re.sults  on  his  local  terminal. 

The  DACS  acquired  a  CD-ROM  reader  and  established  on-line  acce.ss  to  the  Defcn.se  Technical 
Information  Center  (DTIC)  as  a  mechanism  for  acquiring  SERF  data.  In  addition,  the  DACS 
obtained  NASA  ASTRO  CD,  a  CD-ROM  prototype  of  NASA's  Aerospace  and  Technical  Re.search 
database.  The  CD-ROM  contains  five  years  (1988-1992)  of  bibliographic  citations  from 
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NASA/RECON  tiles.  Re.search  projects  can  be  identified  by  DACS  staff  from  NASA  ASTRO 
CD. 

A  Defense  Research  On-Line  System  (DROLS)  account  was  established  with  DTIC.  This  account 
permits  DACS  personnel  direct  access  to  SERF  information  held  at  DTIC  via  modem.  Through 
this  mechanism,  SERF  information  can  be  down-loaded  and  stored  on  the  DACS  computer  system 
for  further  review  and  proce.ssing. 

Flans  currently  exist  to  design  an  on-line  SERF  database  integrated  with  the  other  DACS  STINFO 
databases.  A  database  schema  for  an  on-line  SERF  databa.se  was  designed  early  in  the  contract. 
Additional  STTF  information  will  be  acquired  via  a  survey  of  DoD  contractors  and  sponsors. 
Several  add'  jnal  sources  of  SERF  data  were  identified; 

•  Survey  responses  direct  from  the  contractor  or  sponsor. 

•  14(X)  Work  Unit  Summaries  available  from  the  Defense  Technical  Information  Center 
(DTIC). 

•  Approximately  370  U.S.  universities  recorded  in  the  DACS  U.ser  Frofile  Databa.se, 

•  Approximately  3,250  contacts  at  874  organizations  with  2,154  projecLs  recorded  in  the  DACS 
Robotics  and  Artificial  Intelligence  Database  (RAID), 

•  Information  contained  in  How  to  Get  it  —  A  Guide  to  Defense- Related  Information  Resources 
(DTIC,  AD-A201  6(K)),and 

•  Information  obtainable  from  the  Defen.se  Gateway  Information  System  (DGIS). 


3.3  Software  Engineering  Tools  Information  Database 

The  Software  Engineering  Tool  Information  (SETI)  Database  is  a  eoliection  of  material 
containing  descriptive  information  on  software  engineering  tools,  including  tool  function,  host 
computer,  target  computer,  availability/control,  and  supporting  dix;umentation.  The  SETI  can 
assist  managers  in  locating  tools  of  interest  and  availability,  and  in  identifying  tools  with  similar 
functions  and  features. 

The  SETI  is  implemented  as  a  collection  of  hardcopy  reports,  vendor-supplied  product  literature, 
and  automated  tools  acquired  from  external  sources.  Two  of  the.se  reports  were  supplied  by  the 
previous  operator  of  the  DACS.  These  reports  are  the  Software  Life  Cycle  Tools  Directory  (IIT 
Research  Institute,  March  1985)  and  A  Directory  of  Air  Force  Ada  &  Jovial  Softw  are 
Engineering  Tools  (IIT  Re.search  Institute,  February,  1988). 

In  October  1989,  Kaman  Sciences  Corporation  met  with  the  Software  Technology  Support 
Center  (STSC)  to  discuss  cooperative  efforts  for  collecting  software  tool  information.  STSC 
personnel  explained  their  current  projects  and  the  information  they  hoped  to  collect.  Resources 
available  through  the  DACS  were  explained  to  the  STSC.  It  was  determined  that  the  STSC  had 
greater  resources  than  the  DACS  and  a  charter  to  cover  software  tools.  Since  the  SETI  database 
was  not  contractually  required  for  the  DACS  and  to  prevent  duplication  of  effort,  it  was 
determined  at  that  time  that  the  DACS  would  not  actively  collect  tool  information,  although  it 
would  cooperate  with  the  STSC.  Sub.sequently,  articles  appeared  in  the  DACS  Newsletter 
describing  STSC  activities,  and  the  DACS  acquired  STSC  reports  de.scrihing  .software  tools. 

The  decision  to  reduce  the  emphasis  on  software  tool  data  collection  was  revisited  as  a  result  of 
several  DACS  Technical  Area  Tasks  (TAT)  related  to  the  Army's  Software  Test  and  Evaluation 
Panel  (STEP)  activities.  The  studies  required  the  DACS  to  identify  software  tools  supporting 
STEP  metrics.  Acquiring  information  from  the  STSC  with  this  specific  orientation  proved  very 
difficult.  Accordingly,  the  DACS  began  updating  the  SETI.  The  initial  results  for  STEP  were 
documented  in  A  Survey  of  Tools  for  STEP  Metric  Data  Collection  (Kaman  Sciences 
Corporation,  March  19,  1992). 

Since  then,  additional  tool  information  has  been  acquired  by  the  DACS.  Software  Analysis  and 
Test  Technologies  (Kaman  Sciences  Corporation  and  Re.search  Triangle  Institute,  February 
1992),  a  DACS  Critical  Revicw/Technology  A.s,se,ssment,  lists  information  on  .software  testing 
and  analysis  tools.  Additional  tool  information  can  be  found  in  Testing  Tools  Reference  Guide,  a 
technical  report  acquired  from  Software  Quality  Engineering.  Product  announcements  and 
reviews  contained  in  CASE  Outlook,  a  quarterly  newsletter  to  which  the  DACS  subscribes,  are 
available.  In  addi  .m,  the  DACS  acquired  Tool  Finder/Plus,  a  robust  PC-ba.sed  daiaba.se  of 
Computer  Aided  Softvare  Engineering  (CASE)  tool  descriptions,  produced  by  the  CASE 
Consulting  Group,  Inc. 

Planning  began  to  consolidate  DACS  .software  tool  information  into  an  on-line  database 
integrated  wil'a  the  other  DACS  STINFO  databa,ses.  More  tool  information  will  be  acquired  via 
a  vendor  survey  during  the  implementation  of  that  daiaba.se.  Several  potential  providers  and 
sources  of  SETI  data  will  be  explored; 

•  Approximately  .^71  U.S.  universities  recorded  in  the  DACS  U.ser  Profile  Databa.se 
(UPD). 

•  Vendor  data  in  the  form  of  surveys  and  product  literature  currently  in  the  po.s.se.ssion  of 
the  DACS  for  approximately  25  vendors. 
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•  Data  files  for  ToolCase,  a  repository  of  Computer-Aided  Software  Engineering  Tools, 
produced  by  Tenessee  Technological  University,  and 

•  Information  contained  in  How  to  Get  it  --  A  Guide  to  Defense-Rfloted  Infonnution 
Resources  (DTIC,  AD-A201  6tH)). 

Figures  3.3. 1  through  3.3.4  contain  .several  examples  of  tool  information  found  in  the  SETl. 


Program  Name;  The  Analysis  of  Complexity  Tool 
Acronym;  ACT 
Price;  Unknown 
H  a  rd  ware/So  f  l  ware  P I  a  t  fo  rm  s ; 

Sun,  Apollo,  HP,  NCR  Tower  work.stations,  DEC  VMS  and  Ullrix  systems,  and  the 

IBM  PC 


Description; 

ACT  automatically  pinpoints  where  software  is  too  complex  to  be  reliable  and  quantifies  the 
number  of  tests  needed.  This  tool  produces  How  graphs,  complexity  metrics,  annotated  listings, 
test  paths,  and  test  conditions  "for  a  manyear  of  code  in  under  8  minutes  on  an  IBM  AT."  ACT 
u.nalyzes  C,  FORTRAN,  COBOL,  Ada,  Pa.scal.  various  machine  languages,  CMS-2.  VAX 
Macro,  and  design  languages.  It  automates  the  McCabe  Structured  Testing  methodology 
described  by  the  National  Bureau  of  Standards  Publication  500-99,  the  ba.seline  testing 
methodology  taught  in  the  McCabe  Structured  Testing  Seminars,  and  the  calculation  of 
cyclomatic  complexity. 


Figure  3.3.1 

The  Analysis  of  Complexity  Tool  (ACT) 
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I'rogram  Name:  ACT/Instrumentation 
Acronym:  ACT/lnstrumenlalion 
Price:  K':  S10,500.  Workstation:  $26.5(X)  (single  user) 

Hardware/Soflware  Platlonns: 

Sun,  HP,  1BM(RS-6(XX)),  PC  IX)S.  DPC.  S(il 

Description: 

The  Analysis  of  Complexity  Tixil  and  the  McCabe  instrumentation  fool  guide  an  user  through  the  testing  effort, 
ensuring  that  one  tests  not  only  the  sections  of  code  one  knows  to  be  changed,  but  also  the  sections  of  code  that  one 
might  not  realize  were  affected  tiy  a  change.  The  Analysis  of  Complexity  fool  is  a  dynamic  version  of  the 
Battlemap  Analysis  Tool  and  it  also  specifically: 

1)  Produces  the  Cyclomatic  Metric  (number  of  unit  tests). 

2)  IVoduces  design  complexity  metric  (number  of  integration  tests). 

3)  IVtxluces  the  c.s.sential  complexity  meu-ic  (measure  of  structuredness) 

4)  Produces  actual  complexity  (ctxle  coverage). 


Figure  3  J.2 
A  CT/Iiistr  umenta  tion 


Ih'ogram  Niune:  Ada  Measurement  and  Analysis  Tool 
AcTonym:  ADAMAT 

Price:  S  l(),0f)0  and  up  (Varies  with  Platform) 

Ihirdware/Software  Plalfonns' 

VAX/VMS  (any  VAX),  Rational,  SUN,  PC's  and  UNIX 

Description: 

ADAMAT  provides  a  highly  cost  effective  way  for  .software  engineering  organizations  to  acquire  quantitative 
information  for  detecting  quality  problems  during  the  design,  implementation,  testing  and  operational  phases. 
ADAMAT  provides  comprehensive  and  objective  visibility  into  Ada  program  development,  and,  therefore,  offers 
greater  assurance  of  the  application  of  gixxl  software  engineering  practices  in  Ada;  of  meeting  contract  or  company 
established  programming  .standards;  of  maximum  utilization  of  the  important  powerful  features  of  Ada. 


Figure  3  J.3 

Ada  Mea.surement  and  Analysis  Tool 


I’logram  Name:  AdaQuesi 
Acmnym:  AdaQuest 
Price:  $10,000  +  (depends  on  plailorm) 
Hardware/Software  Platforms: 
DEC  VAXA'MS 


Description: 

AdaQuest  is  a  set  of  integrated  computer-based  software  tools  which  provide  computer  program  test  and  verification 
support  for  the  Ada  programming  language.  AdaQuest  provides  automated  a.ssistance  during  the  (\xlc  and  Unit 
Testing,  CSC  Integration  and  Testing,  CSCl  Testing,  and  Maintenance  phases  of  the  software  life  cycle. 

AdaQuest  performs  Ada  source  code  processing,  including  lexical,  syntactic,  and  semantic  analysis.  Ada  source 
code  that  is  not  in  accordance  with  MII.-STD-I815A  is  identified  by  AdaQuest  and  must  be  corrected  prior  to 
further  analysis.  Once  the  Ada  source  code  is  syntactically  correct,  vtirious  static  and  dynamic  analy.ses  are  available. 
Static  analysis  includes  (1)  multi-mode  symbol  usage  -l<x:ate.s  the  definition  and  u.scs  of  every  symbol  denoting  an 
explicitly  declared  entity,  (2)  program  unit  nesting,  (.^)  call  dependency.  (4)  task  tennination  dependency,  and  (S) 
static  task  analysis  to  identify  potential  circular  deadUxrk. 

For  dynamic  analysis,  runtime  execution  data  is  collected  by  AdaQuest-inserled  software  instrumenuition  probes. 
This  runtime  information  is  then  used  for  subsequent  post-execution  coverage,  timing,  and  tasking  activity  analysis. 
As  part  of  dynamic  analysis,  AdaQuest  provides  for  the  translation  of  manually  entered  "assertions"  into  executable 
code.  Assertions  are  functional  descriptions  which  serve  to  validate  correct  program  functioning. 


AdaQuest 
Figure  3.3.4 
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4.0  DATA  ACQUISITION 


A  major  part  of  the  DACS'  mission  is  the  collection,  storage,  dialysis,  and  dissemination  of 
software  process  and  product  metric  data.  Examples  of  such  metrics  include  fault  density,  failure 
rate,  productivity,  .schedule  length,  complexity  metrics,  and  various  quality  measures  such  as  the 
average  effort  needed  to  correct  a  fault.  Software  metric  data  is  needed  to  validate  certain  models 
intended  to  aid  .software  managers.  Co.st,  size,  reliability,  and  quality  models  have  all  been 
propo.sed  to  assist  in  predicting  and  controlling  various  a.specLs  of  the  software  development 
process  and  characteristics  of  the  resulting  products.  Software  metric  data  is  also  needed  to 
examine  the  quantitative  impacts  of  new  software  technologies  such  as  Computer  Aided 
Software  Engineering  (CASE)  tools.  Object  Oriented  programming,  rapid  prototyping,  Ada,  etc. 
Finally,  software  metric  data  is  useful  for  individual  organizations  in  optimizing  their  proce.ss. 

The  DACS  serves  as  a  central  repository  of  software  metric  data,  thereby  enabling  both 
researchers  and  practitioners  throughout  the  Defense  community  to  advance  both  the  state  of  the 
art  and  the  .state  of  the  practice  in  .software  development  and  maintenance.  Software  metric  data 
is  stored  in  the  Software  Life  cycle  Empirical  Databa.se  (SLED).  The  metric  data  acquisition  and 
analysis  aspects  of  the  DACS  mission  have  always  been  a  chronic  problem  due  to  the  limited 
availability  of  significant  amounts  of  current  data.  A  comprehensive  data  collection  and  analysis 
program  is  characteristic  of  level  4  (Managed)  and  5  (Optimizing)  software  organizations  in  the 
Software  Engineering  In.stitute  (SEl)  proce.ss  maturity  organization.  Early  results  from  an  SEI 
survey  found  that  over  85%  of  .several  dozen  organizations  studied  were  level  1  (Initial),  with  no 
organization  attaining  a  level  4  rating  or  above  (Humphrey  88J.  This  result  demonstrates  how 
rarely  software  organizations  collect  the  data  the  D/.CS  need.s. 

Despite  these  inherent  handicaps,  the  data  acquisition  program  exhibit  some  definite 
achievements.  A  dataset  of  simulated  test  data  was  created  for  examining  the  performance  of 
software  reliability  models.  T4ie  DACS  has  worked  clo.sely  with  some  organizations  initiating 
software  measurement  programs: 

•  The  American  Institute  of  Aeronautics  and  Astronautics  (AI A.^)  Space-Ba.sed 
Observation  System  Committee  on  Standards  Software  Reliability  Working  Group 

•  The  RL  Software  Quality  Technology  Tran.sfer  Con.sortium  (SQT2C) 

•  U.S.  Army  Software  Test  and  Evaluation  Panel  (STEP) 

•  Air  Force  Space  Command  (AFSPACECOM) 

Our  investment  in  supporting  the.se  programs  resulted  in  a  small  amount  of  new  metric  data 
under  this  contract  and  should  result  in  the  acquisition  of  large  amounts  of  data  in  the  future. 

In  addition,  the  DACS  continues  to  di.siribuie  SLED  data.sets.  Procedures  have  been  put  in  place 
to  allow  clerical  personnel  to  prepare  copies  of  the  most  frequently  demanded  datasets.  One  of 
these  data.sets  is  a  major  update  of  the  National  Aeronautics  and  Space  Administration  (NASA) 
Software  Engineering  Laboratory  (SEL)  data.set. 
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4.1  The  Simulated  Software  Reliability  Dataset 

A  recent  DACS  Data  Analysis  Report  describes  an  investigation  of  software  reliability  models. 
The  empirical  portion  of  that  investigation  relied  on  a  simulation  methodology  developed  by  the 
DACS.  This  simulation  methodology  allows  the  examination  of  software  reliability  models 
under  known  conditions.  The  DACS  has  used  this  methodology  to  examine  the  behavior  of 
exponential  class  software  reliability  models  while  varying  the  quality  of  debugging  and  other 
parameters.  The  result  of  this  experiment  was  to  demonstrate  that  typical  problems  observed  in 
applying  software  reliability  models  to  real-world  data  can  arise  when  all  model  as.sumptions  are 
fulfilled. 

The  data  used  in  this  experiment  is  known  as  the  Simulated  Software  Reliability  Dataset.  It  was 
generated  by  simulation  and  consists  of  the  five  files  li.sted  in  Table  8-1.  Because  the  data  was 
generated  for  a  specific  experiment  and  its  general  applicability  may  be  doubtful,  the  Simulated 
Software  Reliability  Data.set  is  not  distributed  by  the  DACS,  although  it  is  described  in  the 
DACS  Data  Compendium  [Kaman  921. 


FILE 

DESCRIPTION 

#  OF  RECORDS 

Simulator  Data 

Simulator  Inputs 

.^(K) 

Test  Data 

Simulator  Outputs 

I57,5(H) 

Program  Data 

Simulator  Outputs 

MK) 

Parameter  Data 

Goel-Okumoto  Ideal 

12 

Parameters 

Model  Data 

Goel-Okumoto  E.stimates 

MK) 

Figure  4- 1 : 

Simulated  Software  Reliability  Data.set  Files  and  Records 


The  Simulator  Data  file  contains  the  parameters  describing  each  program  whose  testing  was 
simulated.  A  formal  experiment  design  was  used  with  twelve  different  cases  of  parameters. 
Within  each  case,  25  programs  were  simulated.  Although  the  same  parameters  described  each  of 
these  25  programs  in  a  ca.se,  the  programs  varied  in  the  exact  location  of  bugs,  the  selection  and 
order  of  test  points  input  to  the  program,  and  the  errors  made  while  debugging. 

The  Test  Data  file  consists  of  the  simulated  data.  Basically,  this  data  is  a  list  of  the  results  of 
running  525  tests  on  each  program.  A  result  can  be  either  a  succe.ss  or  a  failure.  One  can  apply 
almost  any  software  reliability  model  to  this  data. 
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The  Program  Data  file  is  also  data  output  by  the  simulator.  This  file  eoniains  parameters  such  as 
the  number  of  bugs  in  the  program  and  the  number  of  inpuLs  that  trigger  bugs  at  the  end  ol 
testing.  One  can  use  this  data  to  examine  the  accuracy  of  software  reliability  model  estimates. 

A  prototypical  exponential  class  software  reliability  model,  the  Goel-Okumoto  model,  was 
applied  to  the  simulated  test  data  during  the  course  of  the  experiment  described  in  the  data 
analysis  report.  The  Parameter  Data  file  contains  ideal  values  for  the  model  parameters.  These 
values  are  based  on  the  parameters  input  to  the  simulator  and  a  specific  theory  for  applying  the 
Goel-Okumoto  model  under  imperfect  debugging. 

The  last  file,  the  Model  Data  file,  contains  the  results  of  applying  the  Goel-Okumoto  model  to 
the  test  data.  Not  all  values  in  this  file  existed  for  all  programs  tested;  this  condition  is  indicated 
by  a  field  in  the  file.  The  existence  of  parameter  estimates  was  ba.sed  on  the  time  between 
failures,  where  time  is  measured  in  terms  of  the  number  of  successful  tests  between  failures. 
Parameter  estimates  were  actually  calculated  based  on  failure  count  data;  the  number  of  failures 
occurring  in  every  group  of  fifteen  tests  was  input  to  the  model.  How  well  the  model  fit  the  data 
was  asses.scd  by  the  Kolmogorov-.Smirnov  goodness-of-fit  staii.siic. 


4.2  AlAA  Software  Reliability  Data 

In  early  1988,  the  AIAA  held  the  first  meeting  of  the  Space- Ba.sed  Observation  System 
Committee  on  Standards  (SBOS  COS).  The  SBOS  COS  is  developing  standards  that  will  help 
improve  reliability,  safety,  and  reusability  while  reducing  life  cycle  costs  for  space-ba.sed 
missions,  both  manned  and  unmanned.  The  AIAA  has  six  approved  SBOS  projects  underway, 
including  a  Software  Reliability  Working  Group. 

The  objective  of  the  Software  Reliability  Working  Group  is  the  identification  or  development  of 
viable  software  reliability  models  that  permit  quantitative  as.sessment  of  ri.sk  and  prediction  of 
failure  rates.  The.se  models  will  enhance  the  precision  and  consistency  of  the  aerospace  indu.stry’s 
ability  to  compute  the  projected  contribution  of  .software  to  the  reliability  of  its  software 
intensive  systems. 

The  AIAA  has  established  a  Blue-Ribbon  Panel  to  .study  i.ssues  in  this  controversial  area 
comprised  of; 

•  William  Farr,  Ph.D.,  Naval  Surface  Warfare  Center 

•  Allen  L.  Hankinson,  NIST 

•  Herbert  Hecht,  Ph.D.,  SoHar 

•  Myron  Lipow,  Ph.D.,  Hughes  Aircraft 

•  John  D.  Musa,  AT&T 

•  Andrea  Federoff  Sebera,  SAIC 

•  Victor  Selman,  Sc.D.,  American  University 
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Marlin  L.  Shooman,  Ph  D.,  Polytechnic  University 
David  Sieferi,  NCR 


Additionally,  the  following  people  have  made  substantial  contributions  to  this  effort: 

•  Stephen  Kelly,  Kaman  Sciences  Corporation 

•  Devon  Smith,  General  Dynamics 

•  Ted  Keller,  IBM  Houston 

•  Michael  Lyu,  Ph.D.,  JPL 

•  George  Stark,  MITRE 

•  George  Schick,  Ph.D.,  Aerospace  Corporation 

•  Frank  Yap,  Lockheed 

The  working  group  is  developing  several  products,  namely,  a  .software  reliability  handbook,  tool 
evaluations,  comparative  studies  of  reliability  data  .sets,  and  a  software  ru  liability  dataha.se. 

Stephen  Kelly  of  the  DACS  has  actively  participated  in  the  development  of  the  software 
reliability  database.  This  participation  includes  the  writing  of  .several  draft  design  documents  and 
the  development  at  the  DACS  of  a  prototype  implementation  of  the  diiiahase.  When  the  databa.se 
is  populated,  it  will  contain  a  .significant  compilation  of  field-operational  software  failure  data. 
DACS  participation  has  already  resulted  in  the  acquisition  of  hard  copies  of  collections  of 
Software  Problem  Reports  from  several  companies.  More  importantly,  the  AlAA  plans  for  the 
DACS  to  manage  the  software  reliability  databa.se.  Hence,  as  the  AlAA  acquires  software 
reliability  data  in  the  coming  years,  it  will  be  stored  at  the  DACS. 


4.3  Software  Quality  Technology  Traasfer  Consortium  Data 

The  Rome  Laboratory  (RL)  Software  Quality  Technology  Transfer  Cim.sortium  (SQT2C)  is  a 
consortium  of  industrial  orguni/.atitins  with  the  goal  of  transitioning  software  quality 
measurement  into  practice,  particularly  the  technology  as.sociated  with  Rome  Laboratory 
Software  Quality  Framework  (RLSQF).  Through  a  .series  of  special  studies,  the  DACS  has 
helped  RL  begin  the  SQT2C.  The  DACS  participation  in  the  consortium  support  team  has 
already  resulted  in  the  acquisition  of  some  quality  metric  data.  During  the  following  contract,  the 
DACS  plans  on  acquiring  a  va.stly  expanded  amount  of  data  from  this  source. 
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4.4  STEP  Metric  Data 


The  U.S.  Army  Sol'lware  Test  and  Evaluation  Panel  (STEP)  initiatives  are  designed  to  improve 
the  software  test  and  evaluation  priKess  and  implement  a  standard  si.‘i  of  software  metrics.  STEP 
was  initiated  in  September  1989  by  the  U.S.  Army  Operational  Test  and  Evaluation  Command 
(OPTEC).  STEP  has  developed  an  implemented  three  produeLs; 

•  A  standard  set  of  .software  metrics,  .supported  by  a  centrali/ed  metrics  database 

•  A  new  Army  Pamphlet  (DA  PAM  7.^-1,  Volume  6)  with  an  improved  software  test  and 
evaluation  prtK‘e.ss  and  procedures  to  implement  the  metrics  .set 

•  Requirements  for  a  new  d(x;ument.  the  User  Functional  Description  (UFD),  to  better 
capture  u.ser  needs  for  .software  development  projects. 

The  metrics  effort  is  supported  by  the  development  of  a  central  database  for  Army-wide  metrics 
data  collection.  The  Army-wide  database’  will  be  used  to  track  both  the  status  of  Army  software 
and  the  u.se  of  the  STEP  metrics  .set.  The  database  will  al.so  serve  as  the  baseline  for  validation 
and  improvement  of  the  initial  metric  .set.  Through  a  .series  of  special  studies,  the  DACS  has 
provided  support  to  OPTEC  in  beginning  the  STEP  program.  We  have  developed  a  itrol  for 
analyzing  STEP  metric  data,  hosted  a  workshop  for  training  in  the  use  of  .STEP  products,  and 
begun  .setting  up  an  information  clearinghou.se  for  .STEP  information  and  products.  As  a 
consequence  of  this  investment  under  the  current  contract,  the  DACS  anticipates  storing  the 
STEP  metrics  databa.se  when  it  is  created  during  the  following  DACS  contract. 


4.5  Air  Force  Space  Command  Data 

Kaman  Sciences  provides  engineering,  technical,  and  software  support  to  the  Air  Force  Space 
Command  (AFSPACECOM).  Currently,  the  .Sy.stems  and  Software  directorate  of  Kaman 
Sciences,  which  supports  AFSPACECOM,  is  upgrading  their  SEl  software  process  maturity 
rating.  As  a  consequence,  they  are  intere.sted  in  the  use  of  .softw  are  measurement  technology  to 
detect  areas  for  improvement  in  both  their  proce.ss  and  products.  Through  an  Internal  Re.search 
and  Development  (IR&D)  effort,  Kaman  Sciences  has  funded  the  DACS  to  demonstrate  .some 
measurement  technology. 

This  IR&D  has  resulted  in  the  collection  and  analysis  of  certain  static  .source  code  metrics.  This 
analysis  is  documented  in  the  last  data  analysis  report  produced  under  the  current  DACS  contract 
and  further  described  in  Section  5..^.  This  preliminary  .study  should  re.sult  in  the  acquisition  of 
large  amounts  of  .software  metric  data  under  the  following  contract. 

References 
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5.0  DATA  ANALYSIS 


Three  reports  were  produced  summan/.ing  the  data  analysis  activities  under  this  contract.  The 
report  Linear  Software  Reliability  Models  Under  Imperfect  Debugging  jVienneau  91 J  describes 
an  experiment  in  which  simulated  software  test  data  was  used  to  examine  the  behavior  of 
software  reliability  models  under  controlled  conditions.  DACS  Data  Analysis  Papers  [Kaman 
92al  is  a  compilation  9'  technical  presentations  and  papers  given  by  DACS  personnel  relating  to 
technologies  that  provide  the  software  engineering  foundation  of  the  DACS  data  analyses 
program.  SPADC)C  Source  Code  Metrics  Application  j  Kaman  9.^1  compares  and  contrasts 
automatically  collected  source  code  metrics  from  an  Air  Force  Space  Command 
(AFSPACECOM)  with  metric  information  from  three  other  FORTRAN  systems. 


5.1  Linear  Software  Reliability  ModeLs  Under  Imperfect  Debugging 

The  report  Linear  Software  Reliability  Models  Under  Imperfect  Debugging  describes  the  results 
of  an  experiment  designed  under  the  previous  contract  |Vienneau  X9I.  That  experiment 
examined  the  behavior  of  software  reliability  models  under  the  assumption  that  analysts  commit 
errors  while  debugging  software  after  testing  has  detected  a  fault.  A  variety  of  software 
reliability  models  have  been  developed  over  the  la.st  two  decades  to  assist  in  the  specification, 
measurement,  certification,  and  control  of  software  reliability.  A  body  of  knowledge  is  growing 
on  how  well  the.se  models  can  be  expected  to  perform  in  practice.  Hmpirical  applications  have 
uncovered  various  problems.  Numerical  methods  for  parameter  estimates  frequently  diverge. 
Parameter  estimates  can  .seem  quite  different  from  true  values.  Performance  measures,  such  as 
reliability,  can  be  very  variable.  This  study  was  designed  to  investigate  the  cau.ses  of  these 
difficulties. 

Fixploring  the  full  implications  of  software  reliability  modeling  assumptions  and  their  violations 
pre.sents  great  analytical  difficulties.  Since  many  a.s,sumption.s  deal  with  unob.servable,  empirical 
work  is  also  problematic.  Thus  the  opportunity  for  simulation  ari.ses.  This  study  therefore 
adopted  simulation  methodology  that  was  prcvitiusly  employed  for  exploring  the  effects  of 
relationships  between  program  functions,  the  locations  of  bugs,  and  the  selection  of  test  ca.ses  on 
.software  reliability  modeling.  The  study  u.sed  this  simulation  methodology  to  explore  the  effects 
of  variations  in  the  locations  of  bugs,  imperfections  in  debugging,  and  random  testing  on  the 
performance  of  a  repre.sentative  software  reliability  model  when  all  a.ssumptions  are  met. 

The  major  conclusion  of  this  study  is  that  the  problems  ob,ser\’ed  in  practice  when  applying 
software  reliability  models  can  uri.se  even  under  the.se  ideal  conditions.  Programs  in  which  each 
bug  has  a  small  impact  on  the  failure  rate  can  generate  data  for  which  the  equations  defining 
maximum  likelihood  estimates  fail  to  have  a  .solution.  Parameter  estimates  lend  to  be  biased  with 
the  bias  increasing  for  low  reliability  programs.  Errors  in  parameter  estimates  tend  to  cancel  out 
yielding  more  accurate  estimates  of  reliability,  thus  providing  empirical  support  for  the  assertion 
that  reliability  models  shot. Id  be  u.sed  to  control  reliability,  not  bug  content.  Finally,  a  previously- 
proposed  method  of  measuring  goodne.s,s-of-fit  was  .shown  to  be  very  inaccurate. 

Whether  or  not  the  perfcirmance  of  models  with  more  realistic  assumptions  might  be  belter  is  a 
question  that  future  re.search  conducted  with  this  .simulation  methodology  can  explore.  Likewise, 
one  can  examine  the  less  than  ideal  condiliiins  where  some  assumptions  are  violated.  Finally,  the 
performance  of  other  estimation  methods  and  goiidne.s.s-of-fii  measures  should  be  explored.  The 
results  of  this  study  are  pre.senied  in  detail  in  [Vienneau  91  j. 


5.2  DACS  Data  Analysis  Papers 


As  of  July  1992,  DACS  core  personnel  had  presented  six  papers  under  this  eomraet  irealine 
software  metries,  eost  models,  reliahility,  and  quality  measurement.  The.se  measuremem  and 
modeling  technologies  provide  the  software  engineering  foundation  lor  the  DACS  data  analysis 
program.  The  second  data  analysis  report  collected  these  papers  under  one  cover  |Kaman  92al. 
Each  .set  of  slides  in  the  report  is  accompanied  by  a  section  de.scribing  the  oral  deliveries  given  at 
the  conferences  and  meetings  at  which  the.se  papers  were  pre.sented. 

The  data  analysis  report  contains  the  following  six  papers; 

1.  Robert  Vienneau,  "STEP  Metrics  Compared  with  Previou.s  Programs,"  Proceedings  of  the  .^rd 
Annual  Software  Quality  Workshop;  Alexandria  Bay,  New  York;  August  11-15.  1991. 

2.  Robert  Vienneau,  "The  Cost  of  Testing  Software."  Proceedings  of  the  Annual  Reliability  and 
Maintainability  Symposium;  Orlando.  Florida;  January  29-31,  1991. 

3.  Stephen  Kelly,  "Software  Quality  Measurement,"  Software  Improvement  Conference; 
Wa.shington.  DC;  November  29-30,  1990. 

4.  Robert  Vienneau,  "Models  for  Incorporating  Software  into  System  Reliability,"  Society  of 
Automotive  Engineers  AS-5  Committee  Meeting;  Santa  Barbara,  California;  July  16-18,  1990. 

5.  Robert  Vienneau,  "A  General  Poi.s.son  Type  Software  Reliability  Model,"  AlAA  Working 
Group  on  Software  Reliability;  May  1990. 

6.  Stephen  Kelly,  "RADC  Work  in  .Software  Reliability,"  AlAA  Working  Group  on  Software 
Reliability;  November  1989. 

The  DACS  has  pre.sented  one  other  paper  dealing  with  software  engineering  data  analysis  since 
the  preparation  of  this  data  analysis  report; 

•  Robert  Vienneau,  "The  Consolidated  Experience  Factory;  An  Approach  for 

Instrumenting  System  Engineering,"  Proceedings  of  the  1992  Complex  Systems 
Engineering  Synthesis  and  As.se,s,sment  Technology  Workshop,  Naval  Suii'ace  Warfare 
Center,  Silver  Spring,  Maryland,  20-24  July  1992. 

This  last  paper,  "The  Consolidated  Experience  Factory,"  presents  an  approach  for 
instrumentation,  data  collection,  analysis,  and  improvement  of  the  .sy.stems  engineering  process. 
This  approach,  the  Con.solidated  Experience  Factory  (CEF),  has  been  developed  by  Victor  Basili 
of  the  University  of  Maryland  in  conjunction  with  the  DACS.  The  CEF  was  defined  for  software 
development  and  maintenance,  but,  as  the  paper  shows,  the  approach  is  general  enough  to  apply 
to  systems  development  as  a  whole. 

"STEP  Metrics  Compared  with  Previou.s  Programs"  examines  the  software  metrics  developed  by 
Army's  Software  Test  and  Evaluation  Panel.  This  paper  compares  and  contrasts  the  STEP 
metrics  with  other  well  known  metric  programs.  The.se  other  programs  consi.st  of  the  Software 
Engineering  Institute  process  maturity  metric,  the  Air  Force  management  and  quality  indicators. 
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and  ihc  Air  Force  Operational  Test  and  Evaluation  Center’s  program.  This  paper  consists  til  both 
text  and  accompanying  slides. 

"The  Cost  of  Testing  Software  "  presents  a  cost  model  developed  by  the  DACS.  This  cost  model 
is  based  on  any  of  many  .software  reliability  models  and  determines  the  optimal  release  time  for  a 
software  system  based  on  the  minimization  of  lifecycle  costs.  The  cost  model  is  explained  in 
terms  of  certain  ba.sic  economic  principles.  For  this  paper,  the  report  contains  both  text  and  hard 
copies  of  slides. 

The  set  of  slides,  "Software  Quality  Measurement."  presents  technologies,  tools,  and  processes  to 
evaluate  .software  quality.  Technologies  di.scu.s.sed  include  the  RL  Software  Quality  Framework, 
the  Software  Engineering  Institute  proce.ss  maturity  model,  and  various  standards,  including  the 
draft  IEEE  Standard  for  a  Software  Quality  Metrics  Methodology.  Points  of  contact  are  prov  ided 
for  tools  implementing  quality  measurement  technologies.  This  presentation  also  overviews 
.some  RL  .software  quality  initiatives. 

"Models  for  Incorporating  Software  into  System  Reliability"  consists  ol  a  collection  ol  slides 
presenting  a  tutorial  on  certain  sy.stem  reliability  models.  This  talk  began  with  an  overview'  of  a 
military  standard  sy.stem  reliability  program.  The  emphasis  was  on  mtidels  to  aid  in  up  Iront 
design  and  evaluation,  not  on  the  later  testing  that  most  .software  reliability  models  support.  Two 
sy.stem  models  that  explicitly  account  for  .software  were  examined  in  detail.  First,  a  block 
diagram  Markov  model  developed  by  Hughes  was  explained.  Even  for  a  simple  example,  this 
model  quickly  becomes  computationally  difficult.  Second,  a  llowchart-ba.sed  Markov  model  was 
pre.sented.  Both  of  the.se  models  show  how  the  reliabilities  of  individual  components  combine  to 
yield  sy.stem  reliability,  but  the  latter  is  much  more  tractable.  How  to  use  the  second  model  to 
allocate  a  desired  system  reliability  goal  to  component  reliabilities  was  also  di.scu.s.sed.  This  talk 
concluded  with  a  list  of  references  for  those  interested  in  more  details. 

The  American  Institute  of  Aeronautics  and  Astronautics  (AlAA)  Space-based  Observation 
Systems  Committee  on  Standards  Software  Reliability  Working  Group  is  developing  a  .set  of 
recommended  practices  for  .software  reliability.  The  DACS  has  been  actively  participating  in  the 
working  group,  and  the  last  two  papers  in  the  report  are  part  of  that  participation.  "A  General 
Poi.sson  Type  Software  Reliability  Model"  pre.sents  a  single  software  reliability  model  that 
includes  many  previously  developed  models  as  special  cases  or  as  approximations  to  special 
ca.ses.  The  paper  develops  the  model,  shows  how  to  estimate  its  parameters,  and  pre.sents  a 
goodness-of-fit  test. 

The  .set  of  slides,  "RADC  Work  in  Software  Reliability,"  overviews  past  Rome  Laboratory 
re.search  in  reliability,  particularly  software  reliability.  Related  areas,  such  as  quality 
measurement,  are  di.scus.sed.  The  DACS  role  is  al.s()  discussed. 

5.3  SPADOC  Source  Code  Metrics  Application 

Kaman  Sciences  maintains  a  large  software  system,  SPADOC,  I'or  the  Air  Force  Space 
Command  (AFSPACECOM)  at  the  Cheyenne  Mountain  Complex.  Under  this  contract,  we 
collected  certain  source  code  metrics  from  SPADtX?  by  u.se  of  a  FORTRAN  static  analysis  tool, 
the  Source  Analyzer  Program  (SAP).  We  also  began  collecting  di.screpancy  data  on  the 
SPADOC  system,  but  the  di.screpancy  data  was  not  yet  complete  enough  to  be  analyzed  under 


this  contract.  SPADOC  consists  of  1 1 .4()7,()2{)  lines  of  FORTRAN  source  code,  including 
comments  and  blank  lines. 

The  SAP  was  al.so  u.sed  to  collect  metric  data  from  three  reference  systems,  IRSS/PAAS, 
ARCRHF2,  and  MMI.  The  IRSS/PAAS  system  is  a  radar  and  antenna  simulation  system 
developed  for  Rome  Laboratory  beginning  in  the  late  197()s.  The  system  has  undergone  two 
major  ports  and  extensive  functional  enhancements  during  its  lifetime,  reveral  different  teams  of 
engineers  developed  IRSS/PAAS.  The  .system  was  originally  designed  as  a  batch  .system  and 
later  converted  to  an  interactive  .sy.stem.  The  version  of  IRSS/PASS  examined  under  this  study 
consists  of  64,959  .source  lines  of  code. 

ARCREF2  is  another  radar  simulation  .system  developed  for  Rome  Laboratory.  It  was  developed 
in  the  1980s,  and  is  a  more  typical  example  of  a  modern  interactive  FORTRAN  77  system. 
ARCREF2  consists  of  .'^0,034  .source  lines  of  code. 

The  MMI  system  is  a  signal  proce.ssing  lest  bed  with  a  generic  knowledge-ba.sed  application 
generator,  a  Graphical  U.ser  Interface  (GUI)  man-machine  interface,  and  application-specific 
processing  modules.  The  development  of  the  MMI  system  began  in  1990.  The  software  was 
written  in  both  FORTRAN  and  C,  with  the  FORTRAN  code  being  in  a  Ratfor-like  preproces.sor 
language.  Only  the  FORTRAN  .source  code,  as  automatically  generated  by  the  preproces.sor,  was 
analyzed.  The  FORTRAN  portion  of  MMI  con.si.sls  of  81,494  source  lines. 

The  IRSS/PAAS,  ARCREF2,  and  MMI  systems  provided  ba.seline.s  with  which  we  compared 
and  contra.sted  SPADCXT.  The  distribution  of  each  type  of  FORTRAN  .statement  (A.ssignment, 
Block  data.  Call,  Character,  Common,  Continue,  etc.)  was  determined  for  each  sy.stem.  Usage  of 
questionable  practices  was  noted.  For  example,  input  and  output  was  not  re.stricted  to  a  few 
modules  in  the  ARCREF2  system,  indicating  a  potential  maintainability  problem.  SPADCXT  was 
compared  with  each  reference  .system  in  terms  of  the  distribution  of  .selected  metrics.  Modules  in 
SPADOC  that  had  metrics  values  in  the  tails  of  these  distributions,  and  arc  therefore  potentially 
failure-prone  and  difficult  to  maintain,  were  indicated. 

This  data  analysis  study  demonstrated  a  technique  which  can  be  u.sed  to  provide  significant 
insight  into  potential  software  maintenance  problems  for  very  large  .systems.  The  technique 
identifies  particular  modules  which  may  be  error  prone.  The  technique  relies  on  automated  data 
collection  and  as  a  result,  it  is  appropriate  when  a  manual  examination  of  the  source  code  in 
prohibitive  due  to  sy.stem  size. 

The  analysis  method  is  not  specific  to  FORTRAN  or  the  metrics  used  in  this  report.  We  believe 
that  the  methods  developed  in  this  report  are  u.sable  for  an  examination  of  other  large  software 
systems  phasing  into  operations  and  maintenance  as  part  of  the  Cheyenne  Mountain  Upgrade. 

As  a  pan  of  our  .study  we  recommend  that  a  number  of  enhancements  need  to  be  made  to  .SAP  to 
improve  iLs  effect! vene.ss  for  analy.ses  such  as  this  one. 

•  SAP  should  be  made  more  robust 

•  SAP  should  be  modified  to  allow  additional  keywords  to  be  added  dynamically 

•  Several  computed  metrics  need  to  be  verified  for  correctness 

•  Additional  metrics  .should  be  added  to  SAP 
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5.4  Data  Analysis  Activities  Related  to  Special  Studies 

Certain  spec'  studies  included  the  analysis  of  software  metric  data.  Quality  metrics  were 
collected  under  the  Man-Machine  Interface  Experiment  and  examined  for  their  ability  to  predict 
the  effort  required  to  reuse  modules  in  a  new  system  |Kaman  911.  Under  the  Invesiigatii)n  of  the 
Quality  of  Signal  Processing  Software,  automated  quality  metrics  were  collected  and  analyzed 
for  certain  FORTRAN  subroutines  |???).  The  Ada  9X  Implementation  Analysis  Support  study 
included  the  collection  and  analysis  of  some  common  static  source  code  metrics  for  a  couple  of 
Ada  systems  jKaman  92b].  Under  IV&V  Support  for  the  Range  Control  System,  quality  metric 
data  was  collected  and  analyzed  for  the  requirements  pha.se  of  a  large  software  prefect.  The 
metric  data  on  the  IRSS/PAAS  and  ARCREF2  systems  with  which  SPAD(K"  metrics  were 
compared  was  collected  under  the  IRSS/PAAS/ARCREF  study. 

Other  special  .studies,  although  they  did  not  include  metric  data  collection  and  analysis,  relate  to 
the  kind  of  activities  conducted  under  the  data  analysis  function  of  the  DACS.  Along  with  a 
DACS  subcontractor.  Dr.  Amril  Goel  of  Computer  Software  Modeling  Associates,  we  assisted 
the  Army  Operational  Test  &  Evaluation  Command  (OPTEC)  in  creating  a  tool  for  analyzing 
software  metric  data  ic  be  collected  by  the  Software  Te.sl  and  Evaluation  Panel  (STEP)  program. 
This  assistance  was  provided  under  the  following  special  studies; 

•  Prototype  Software  Metrics  Analysis  &  Project  Management  Tool  investigation 

•  Prototype  Software  Metrics  Analysis,  Project  Management,  &  Risk  Analysis 

•  Software  Proce.ss  &  Software  Metrics  User  Functional  Description 

•  Software  Metrics  Tool  Support 

We  a.ssisted  RL/C.^CB  in  initiating  a  program  to  validate  the  Rome  Laboratory  Software  Quality 
Framework,  a  hierarchical  structure  of  .software  metrics,  by  executing  the  following  special 
studies; 

•  Software  Quality  Automated  Method  Validation:  Setup  &  Support  Task 

•  Software  Quality  Lab  Support 

•  RL  Software  Quality  Con.sortium  Support 
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6.0  TECHNICAL  REPORTS 


The  DACS  has  published  a  number  of  state-of-ihe-art  reports  (SOARs),  critical  reviews/technical 
assessments  (CR/TAs),  data  analysis  reports,  and  other  technical  reports  under  this  contract.  The 
reports  are: 

•  Software  Reusability  (a  SOAR) 

•  Distributed  Database  Technology  (a  SOAR) 

•  CASE  Technology  (a  SOAR) 

•  Artificial  Neural  Networks  Technology  (a  SOAR) 

•  Software  Analysis  and  Test  Technologies  (a  CR/TA) 

•  Object  Oriented  Design  (a  CR/TA) 

•  Software  Quality  (a  CR/TA) 

•  Requirements  Engineering  and  Rapid  Prototyping  (a  CR/TA) 

Software  Reusability  is  a  State  of  the  Art  report  prepared  by  Valentin  W.  T'lrman  Jr.,  of 
Productive  Data  Systems  in  Monument  Colorado.  The  report  describes  software  reusability  as  a 
management  problem,  not  just  a  technical  problem.  The  advances  in  US  software  technology 
and  software  reusability  are  slow  at  best.  The  Private  Sector  is  ahead  of  the  DoD  in  its  efforts 
simply  because  the  software  development  process  is  profit  oriented,  and  most  programs  arc  still 
of  manageable  size.  The  question  of  software  reusability  is  a  complex  one  that  affects  all 
vendors  in  software  engineering,  as  well  as  their  clients. 

This  SOAR  addresses  a  variety  of  issues,  problems,  endeavors,  and  advances,  both  in  the  US  and 
overseas.  Section  I  provides  a  report  summary.  .Section  2  looks  at  the  overall  problem  of  what 
to  do  about  reusability  and  includes  a  number  of  opinions  from  industry  leaders.  Section  3, 
reusability  in  the  US,  discusses  software  libraries,  repositories,  tools,  and  issues  of  cost  and 
quality  as  related  to  software  reusability.  Section  4  examines  the  progress  that  the  Pacific  Rim 
countries  and  Europe  are  experiencing.  The  software  factory  approach  currently  being  u.sed 
successfully  in  Japan  is  discus.sed  and  explained  in  some  depth.  Section  5  discusses  some  of  the 
problem  areas  associated  with  software  reusability.  Finally,  conclusions  and  recommendations 
are  presented  that  include  a  discussion  of  factors  and  i.ssues  that  need  to  be  addres.sed  when 
considering  reusability  in  the  development  and  maintenance  of  software. 

Distributed  Database  Technology  is  a  State  of  the  Art  report  prepared  by  Ms.  Carol 
Wawrzusin,  a  member  of  the  DACS  staff.  It  de.scribcs  a  distributed  daiaba.se  as  a  collection  of 
multiple,  logically  interrelated  daiaba.ses  di.siributed  over  a  computer  network.  A  Di.stributed 
Database  Management  System  (DDBMS)  is  a  .software  system  that  permits  the  management  of 
distributed  data  making  the  distribution  transparent  to  the  u.ser.  This  report  reviews  the  issues 
that  arise  with  such  systems,  surveys  current  commercially  available  DDBMS’s,  and  summarizes 
the  state  of  the  art.  Although  no  standards  yet  exist  within  this  new  technology,  some  guidelines 
have  been  provided  by  C.J.  Date,  and  E.F.  Codd.  True  implementations  of  general  purpo.se 
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DDBMS’s  are  only  now  beginning  lo  emerge  in  the  marketplace .  Their  implementations  with 
respect  to  issues  of  distributed  database  technology  differ  markedly. 

CASE  Technology  is  a  State  of  the  Art  Report  prepared  by  Mr.  James  J.  Reed,  DACS  Director. 
The  report  provides  a  generic  definition  of  CASE  as  any  set  of  computer-implemented,  defined 
tools  and  methods  employed  to  assist  in  the  requirements  analysis,  design,  implementation  and  or 
maintenance  of  a  software  system.  While  limited  in  its  scope  by  the  size  of  the  CASE 
technology  area,  the  report  does  provide  a  reference  for  the  software  engineer  and  program 
manager  concerning  the  selection  and  application  of  CASE  technology,  tools  and  methods  to  the 
problems  encountered  in  software  engineering.  The  report  also  di.scusses  the  introduction  of 
CASE  technology  into  an  organization  as  well  future  directions  that  CASE  technology  will  likely 
follow.  The  topic  of  CASE  tool,  technology  and  implementation  methodology  is  also  discussed 
along  with  the  potential  for  confiicts  due  a  lack  of  standardization.  The  institution  of  more 
formalized  methods  allowed  by  CASE,  at  least  internal  to  a  particular  project  is  noted,  however. 

Artiilcial  Neural  Networks  Technology  is  a  Stale  of  the  Art  report  produced  by  Mr.  Dave 
Anderson  and  Mr.  George  McNiell  of  Kaman  Sciences  Corporation.  The  report  is  intended  to 
help  the  reader  understand  what  Artificial  Neural  Networks  are,  how  to  use  them,  and  where  they 
are  currently  being  used.  Touted  as  the  wave  of  the  future.  Artificial  Neural  Networks  are  self¬ 
learning  mechanisms  which  don’t  require  the  traditional  skills  of  a  programmer.  But  the  reality 
is  often  different  from  the  hype  and  developers  have  often  come  to  the  conclusion  that  neural 
networks  are  complicated  and  confusing.  In  examining  several  efforts  and  methods  for 
developing  Artificial  Neural  Networks,  examples  are  cited  and  the  structures  are  provided.  The 
introduction  provides  an  up-front  guide  to  the  reader  who  wants  a  simple  explanation  of  the 
technology  and  a  road  map  for  those  who  want  substantially  greater  details. 

Software  Analysis  and  Test  Technologies  is  a  critical  review/technology  assessment  prepared 
by  staff  members  from  the  Research  Triangle  Institute  and  the  DACS.  The  report  examines 
current  software  analysis  and  test  technologies  and  the  needs  that  should  be  filled  by  future 
technology.  Analysis  and  testing  of  software  includes  all  life  cycle  activities  conducted  lo  verify 
and  validate  the  software  product.  These  activities  are  undertaken  with  the  goal  of  assuring  the 
robustness  of  the  development  process  and  the  integrity  of  the  developed  product  throughout  the 
life  cycle.  Successful  strategies  for  analy.sis  and  test  must  provide  decision  .support  information 
to  the  acquisition  manager,  the  certifying  agent,  and  the  field  engineer.  There  is  a  need  for 
quantitative  empirical  data  to  demonstrate  when  and  where  various  techniques  are  most 
successful.  There  is  a  need  for  integrated  development  environments  which  include  analysis  and 
test  support.  This  report  also  considers  improvements  in  analysis  and  test  required  to  support 
trends  in  formal  methods,  object  oriented  development,  parallel  programming,  and  system 
engineering. 

Object  Oriented  Design  is  a  critical  review/technology  a.s.sessment  report  prepared  by  Mr. 
Robert  Vienneau  of  the  DACS.  The  report  provides  a  basic  understanding  of  Object  Oriented 
Design  (OOD)  and  some  of  its  features.  The  report  briefly  summarizes  the  history  of  OOD, 
includes  a  description  of  an  OOD  methodology,  and  defines  and  discu.sses  various  concepts  and 
terminology  u.sed  in  OOD.  The  level  of  support  that  various  programming  languages  provide  for 
OD  is  discussed  in  some  detail.  Languages  covered  include  Modula-2,  Ada,  C-(-+,  Object  C, 
LISP,  Smalltalk,  and  Eiffel.  Section  4  discu.s.ses  how  OOD  interacts  with  areas  of  current 
software  engineering  research,  especially  software  reuse  and  alternative  life  cycle  models.  The 
report  also  includes  a  glossary  of  OOD  terms  and  an  annotated  bibliography  of  related  papers 
and  reports. 
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Software  Quality  is  a  criiical  roview/icchnology  asscssmoni  prepared  by  Mr.  Valentin  W. 
Tirman,  Jr.  and  Mr.  Gene  Miluk  of  Productive  Data  Systems  in  Monument,  Cttlorado.  The 
report  provides  an  a.ssessmeni  of  the  current  .state  of  the  art  and  practice  concerning  so/ 1  ware 
quality.  The  report  is  a  compilation  of  where  the  technology  is  at  currently.  It  di.scu.s.se.s  the 
management  aspects  of  dealing  with  software  quality  initiatives,  provides  an  overview  of 
significant  work  being  accomplished  in  metrics,  process  improvement  and  describes  work  being 
done  overseas.  The  175  page  report  is  divided  into  12  Sections.  Conclusions  and 
recommendations,  references  and  bibliography  and  computations  are  all  provided  in  the  report. 

Requirements  Engineering  and  Rapid  Prototyping  was  prepared  by  Dr.  Joseph  Urban  of 
Arizona  State  University  and  Mr.  James  J.  Reed,  DACS  Director.  This  critical 
review/technology  asse.s.sment  includes  the  motivation  for  using  .software  prototyping  in  general 
and  specifically  in  the  context  of  requirements  engineering.  An  overview  of  software 
prototyping  covers  life  cycle  models,  approaches,  pitfalls,  and  opportunities.  The  chapter  on 
software  requirements  and  specification  establishes  a  basis  for  investigating  techniques.  A 
comprehensive  and  exhaustive  analysis  of  .software  requirements  and  .specification  techniques 
and  tools  for  prototyping  addres.ses  sixty  techniques  across  a  variety  rrf  application  domains.  The 
analysis  includes  a  summary  of  common  a.spects  among  the  techniques.  Software  technology 
transfer  is  addre.s.sed  in  this  report  from  the  standpoint  of  past  problems,  avenues  of  opportunity, 
and  actual  experience  in  this  area.  The  report  concludes  the  topic  of  prototyping  and 
requirements  engineering  with  an  a.s.se.s.sment  of  practice,  availability,  potential  areas  of  research. 
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7.0  INFORMATION  TRANSFER 

A  primary  activity  of  the  DACS  involved  transferring  information  from  the  laboratory  or  other 
research  environment  to  the  field  where  it  could  be  applied  it)  solve  software  engineering 
problems.  We  also  kept  our  users  informed  about  the  latest  developments  in  the  field  and  the 
standard,  as  well  as  new  products  and  services  available  to  the  users  from  the  DACS. 

To  accomplish  those  information  transfer  goals,  we  have  carried  out  an  active  current  awareness 
campaign  using  the  tools  available  to  us.  They  include  the  following; 

•  Publication  of  a  quarterly  DACS  Newsletter 

•  Publication  of  a  DACS  Bulletin 

•  Participation  in  the  activities  of  national  organizations 

•  Pre.senlations  at  events  involving  software  engineering/lechnology 

•  Development  of  contacts  within  ma  ji)!  .software  engineering-related  organizations 


7.1  DACS  Newsletter 

The  Quarterly  DACS  Newsletter  was  the  subject  of  numerous  favorable  comments  throughout 
the  DACS  comments.  Most  of  those  comments  were  targeted  toward  the  enhancements  in 
formal  and  content  introduced  during  the  last  half  of  the  contract.  The  length  of  the  newsletter 
was  extended  to  accommodate  more  information  and  added  regular  features. 

Most  often  cited  as  favored  articles  were  the  software  engineering  conference  summaries,  the 
“Clo.sc-up  Corner”  section  which  takes  a  more  in-depth  look  at  an  aspect  of  .software  technology, 
and  details  provided  about  DACS  software  engineering  activities.  Also  popular  was  the 
calendar  of  upcoming  events  which  was  u.sed  to  announce  not  only  DACS-sponsored  activities, 
but  other  technical  conferences  and  activities  of  interest  to  the  community.  The  DACS 
Newsletter  is  a  free  product.  It  was  provided  to  more  than  7,(K)0  DACS  u.sers  each  quarter.  A 
concerted  effort  is  being  made  to  make  sure  that  DoD  and  industry  leaders  receive  the  DACS 
Newsletter  and  are  made  aware  of  our  capabilities,  products  and  .services. 


27 


DACS  Newsletter 

Data  &  Analysis  Center  for  Software 


4  Analysis  Center  for  Software  3  258  Genesee  St,  Suite  103  Utica,  NV  13S02  Q  31 5- 734-369< 


DIRECTOR’S  NOTES 

This  month  in  the  DACS  Newslcuer,  we  have  highlighted  the 
acuvities  of  the  U.S.  Amy's  Software  Test  and  Evaluation 
Panel  (STEP)  Workshop,  which  was  held  in  Denver.  Colorado 
in  Fehniary.  Hosted  by  the  DACS.  the  workshop  provided  in- 
depth  coverage  of  the  Amy  Operational  Test  and  Evaluation 
Command's  initiative  to  iniegraie  software  test  and  evaluation 
principles  into  the  software  development  process.  The 
OPTEC  point  of  contact  for  obtaining  further  infomation  on 
the  STEP  initiauve  is  listed  on  page  three. 

Also  note  on  page  three,  the  preliminary  aniKMincement  of  the 
Fourth  Software  Quality  Workshop,  scheduled  for  August  2  - 
6.  1992  and  in  the  Upcoming  Conferences  and  Events  secuon. 
the  announcement  of  KBSE-7.  scheduled  for  September  20  - 
23.  1992.  this  year's  KBSE  conference  will  be  held  in 
Tysons  Corner.  VA.  These  are  very  popular  meeungs  so 
make  your  reservations  early. 

Several  sample  bibliographies  from  the  DACS  Software 
Engineering  Bibliographic  Daubase  have  been  provided  for 
your  csaminauon.  ITie  DACS  has  been  adding  software 
enguieenng  citations  at  a  rate  of  over  3(X)  per  month.  Within 
the  next  several  months  we  will  publish  several  new  "hard 
copy "  volumes  of  the  infomauon  abstracted.  Remuider,  we 
also  perfom  custom  searches  of  our  complete  bibliography  for 
a  small  fee.  If  you  have  further  questions  or  would  like  to 
access  our  bibliographies,  contact  Ms.  Barbara  Radzisz  at  the 
DACS.  315-734-3696, 

In  November  1991  a  Computer  Aided  Acquisition  and 
Logistics  Support  (CALS)  conference  and  exposiuon  was  held 
in  Arizona.  While  CALS  is  not  strictly  a  software 
engineering  concern,  it  does  have  an  effect  on  us  all  as  a  DoD 
mandated  uiitiauve.  The  DACS  supports  the  CALS  uiiuauve 
in  our  involvement  in  several  special  study  areas  and  in  our 
work  with  the  several  DoD  components  involved  with 
standards  and  specificaiioiis  efforts. 

In  our  CLOSE-UP  CORNER  this  month  we  lake  a  look  at 
various  aspects  of  Software  Reliability.  If  you  wish  to 
contribute  an  article  on  software  reliability  or  another  aspect  of 
software  engineering  please  contact  the  DACS. 
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7.2  DACS  Bulletin 


A  vaneiy  ot  DACS  Bulleiins  were  prepared  and  distnbuied  id  a  limned  audience.  -Primarily,  the 
bulleuns  were  provided  lo  Rome  Laboratory  personnel  with  a  small  number  provided  ouLside  the 
Lab  to  subscription  customers  and  selected  others.  The  bulletins  summarized  conlerences  and 
events  within  the  soltware  engineering  community,  described  new  and  interesting  technology 
trom  within  the  community  and  described  on-going  research  ol  interest  to  bulletin  recipients. 
Much  ol  the  utility  ot'  the  bulletin  was  lost  with  the  substantial  e.spansion  of  the  DACS 
.Mewsletter. 

DACS  Bulletin 


i^ACS  Ri.netin 


Volun  (3  IX  Numoef  6 


Jjly  1989 


COMMON  ADA  MISSILE  PACKAGES  UPDATE 


•••‘uy.jDke  subwate  is  •.•tSKn  oisojsseo  arxj  is  a  r-.>or'»v  souont  commoo'iv  in  me  aerospace  community 
■nee  Its  ifx.eoiion  m  i^t34  :ne  ‘>ommon  Ada  M«ssiie  Kaci^aqe  tCAMPt  Protect,  primaniy  tunoeO  by  tne 
vtAPS  Jc  ni  t-’roQfam  sponsorea  ov  tne  A*r  force  A/mameni  Laooratory  ana  penormea  by  me 

McDonnell  OouQias  Missue  Systems  Company  nas  ukco  a  praqmaic  apofoacn  lo  aemonstraiinq  me 
■•.•asiO'iity  ano  utility  of  mis  concopi  tor  teai  nie  emoeooea  systems  Camp  prooucts  inciuoe  4S4 
perationai  tuqni  ironware  parts  m  Aaa  'or  tacticai  nussiies  atmoncs  lo'mameni  eiectroncs 
oenenmarxs  tor  evaluating  Aaa  compaers  ana  a  prototype  parts  eogirteenrx)  system  to  support  pans 
uernitication  cataoqing  ana  ccnsinjciion  ''  o'oer  lo  (JerT>onsiraie  me  watue  ot  tne  reuse  ror'cept.  a 
■nisstie  Subsystem  was  bunt  usma  me  CAMP  pans  Mesufls  if>oicaie  a  sgoitcani  increase  m  soir^^are 
proouctivfiy  wnen  aeveopinq  systems  using  Carts  Aoa.  rTX)aerr>  sonware  engmeenrx)  practices  robust 
oirware  toots  jrxj  KrxDi^ieoqeabte  sonware  ervjineers 

r**nu'rea  lasns  associateo  wan  ?ne  camp  pmiect  are  aesenbea  in  tnree  competibve  comracts 
awaroeo  to  me  McOonneii  Dougias  Missiie  Systems  Cortoanv  CAMP-i  iSeptemper  t984  to  Sepiemotrf 
’385)  'Aas  a  teasibiirty  study  wmeP  resulted  m  tne  design  ot  454  software  pans  ang  a  supponinq  pans 
yrxjmeeruv)  s»s.^?m  camp  2  Seoiemoer  t  .f85  to  May  t089J  was  an  impiementafon  ,inc 
demonstraiiort  etton  resulting  m  tne  initial  0'Stri0ul»0r'  ot  CAMP  software  to  Over  *J50  governn>ent 
jqenoes  arva  contractors  CAMP  'i  ijoiy  i9d8  to  Marcn  !09ti  is  a  refinement  and  teenrKnogy  transition 
I'rogram  lor  me  camp  software  af>o  tecnniQues  aii  CAMP  products  were  developed  m  accordance 
■^'in  oco  sro  2 16; 

A  cJomairi  .ingivsis  Of  rrussne  f'ignt  '■-•oftware  Systems  to  identify  commonality  was  penorrr»eO  during 
^AMP  1  'nis  analysis  iixiudeO  an  categories  ot  missnes  (air  to  air  aif  to  qroono  arx3  grouno  to  ar: 
'''Omficar*  commonality  was  found  Pans  reauiremef^is  specrfications  ano  top  levei  pans  OesKjn  w«  re 
ompieied  ■"<  .idOition  a  soeciticaiKjn  an-d  arcnrieciutai  design  was  oeveiooeo  for  me  Camp  Aoa 
'.tissue  Pans  Unqintiorinq  fcjpen  System  iAMPEBi  Ounng  ims  pnase  o»  me  CAMP  Project  Ine 
duTDose  of  mis  system  »$  to  assist  sottware  engineers  >n  locating,  orxjerstarxjiry;  usirx^,  ana  manaqing 
'AMP  pans 

r'e  design  imDiemeniaion  arx3  testing  ot  tne  AMPEE  arx)  associated  pans  were  completed  Oiifinq 
'.AMP-2  Piignt  soltware  tor  an  actuat  missue  aopficaiion  m  a  real-time  processor  m-me  iooo  simulation 
Aias  oevetoped  during  CAMP  2  m  addition  CAMP  2  tasKs  mcn'Oed  me  use  o*  CAMP  pans  to  develop 
1  set  ot  arrrxjmcs  benenmarvs 

CAMP-3  IS  focusirxj  on  refining  pans,  transiiionifx)  tbe  lecnrx)iogy  to  new  users,  convening  tne  AMPEE 
'0  Ada  on  a  VAX  developing  a  training  manual  for  OeveiopinQ  and  Usmg  Ada  Pans  m  Real-Time 
embedded  Appiicafions,  arxj  producing  an  eaecutive  overview  voeo  describing  software  reuse  issues 
irx]  the  CAMP  Programs  approach  to  these  issues  The  video  win  examine  the  soltware  crisis  ang  me 
epportunitv  It  has  created  (or  Ada  arxj  reusable  software  The  new  ampeE  will  contain  numerous 
enhancemertfs  makirxq  rt  a  suitable  pans  erxjmeenng  system  for  a  small  comoany  or  (or  manaqirxj  pans 
for  a  large  system  irKiuded  m  me  enharx:ed  AMPEE  wiii  be  me  development  ol  a  Meta  Consinjctor 
The  current  AMPEE  has  f2  constructors  used  to  develop  special  pans,  such  as  autopilots  ar>d  Kaiman 
'lifers  The  Mefa-Consfructor  w*ii  be  able  to  develop  ofher  constructors,  reducing  me  intense  efton 
'c-au‘reo  to  consiruciors  The  retmed  parts  are  scheduled  (or  release  in  August  t969  The  rraining 
manual  for  aeveiooir>Q  arxj  using  Ada  pans  ano  me  caiaiog  pon»on  of  the  ampEF  w.h  be  i-efeaseo 
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8.0  PRODUCTS  AND  SERVK  KS 


A  variety  of  traditional  and  new  products  and  services  were  made  available  to  the  DAC'S  users 
In  the  CORE  DACS  area,  they  range  from  free  answers  to  relatively  minor  questions  on 
software  engineering  and  technical  areas  to  more  formalized  solutions  to  difficult  problems 
Many  of  the  reports  and  traditional  products  and  services  are  highlighted  in  other  sections  of  this 
repon. 

DACS  CORE  PRODUCTS  &  SERVICES 

•  TECHNICAL  INQUIRIES 

•  BIBLIOGRAPHIC  INQUIRIES 

•  DATABASE  REPORTS/DATASETS 

•  ST  ATE-OF-TH  F:- A  RT-REPORTS 

•  CRITICAL  REVIEVVS/TECHNOLOGY  ASSESSMENTS 

•  DATA  ANALYSIS  REPORTS 

•  MANUALS/HANDBOOKS/STANDARDS 

•  SOFTWARE  TECHNOLOGY  BRIEFIN(;S 

•  SELECTED  PAPERS/PROCEEDINGS 

•  NEWSLETTERS/SPECIAL  BULLETINS 

•  SPECIALIZED  SOFTWARE  DATA  REPOSITORIES 

•  SIMULATION/MODEL  DEVELOPMENT  &  EV ALUATION 


Technical  Inquiries  were  received  in  a  variety  of  ways  and  processed  daily  by  the  members  of 
the  DACS  Staff.  Some  inquiries  were  for  general  information  on  the  DACS,  products  and 
services,  or  more  specific  in  nature,  such  as  questions  on  software  engineering  environments,  etc. 
To  satisfy  these  inquiries,  the  DACS  response  included  forwarding  one  or  more  of  the  following: 

•  DACS  Information  Package 

•  Customized  Bibliographic  Search 

•  Document  Distribution 


Database  Subset  Production 
Information  Summary 


•  Infiirmalion  Analysis 

•  Rclcrral  lo  Other  Sources 

Bibliographic  Inquiries  were  provided  to  individuals  requesting  searches  of  DACS  data 
holdings  or  as  a  part  of  a  technical  area  task.  Routinely,  a  search  strategy  is  developed  and  the 
results  provided  within  two  days  of  the  request  The  DACS  maintains  the  capability  to  provided 
the  results  in  hardcopy  or  softcopy  form.  Abstracts  may  cover  the  entire  spectrum  of  our 
bibliographic  database  or  they  may  be  time  or  context  limited.  Almost  14,(KK)  references  are 
currently  contained  in  the  software  engineering  bibliographic  databa.se.  Sometimes  information 
was  augmented  by  access  to  our  otner  existing  databases  as  well. 

Database  Reports/Datasets  were  produced  upon  request  from  our  user  community.  Generally 
the  information  was  pertinent  to  the  software  engineering  lifecycle  and  incorporated  information 
on  cost,  complexity,  software  problems,  change  data,  ana  productivity  and  reliability  factors.  We 
also  distributed  information  to  potential  u.sers  to  assist  them  in  determining  their  data.set  needs. 

State-of-the>Art  Reviews  are  technical  reports  on  .software  engineering  technologies  of  major 
interest  to  our  u.ser  community.  They  provide  a  consolidated  look  at  specific  areas  of  the 
technology  in  one  document  that  de.scribes  the  technology  and  include  reference  material  useful 
for  further  study.  Tbe  reports  produced  under  this  contract  by  members  of  the  DACS  staff  and 
our  subcontractors  is  di.scu.s.sed  elsewhere  in  this  report. 

Critical  Reviews/Technology  Assessments  take  a  more  in-depth  look  at  issues  of  .software 
engineering  and  software  technology.  They  provide  .some  insight  into  the  technology  and  a.s.sess 
the  strengths  and  weakne.s.se.s  of  the  technology  and  perhaps,  specific  implementations  of  that 
technology. 

Data  Analysis  Reports  consisting  of  empirical  data  maintained  by  the  DACS  in  our  databa.se.s 
and  da  sets  were  made  available  to  our  u.sers  in  the  form  of  analyzed  data  for  comparison  or 
validation  activitie.s. 

Manuals/Handbooks/Standards  The  DACS  distributed  a  variety  of  specialized  products 
designed  to  bring  procedural  standardization  to  the  software  engineering  community.  We 
distributed  DOD-STD-2167A,  DOD-STD-2168,  and  a  variety  of  other  published  standards  to 
DACS  users.  We  also  prepared  manuals  and  handbooks  specific  to  technical  area  ta.sks  to  system 
and  software  owners  and  u.sers. 

Software  Technology  Briefings  are  provided  to  DACS  u.sers  and  potential  u.sers  which  de.scribe 
the  capabilities  of  the  DACS  staff  and  our  subcontractors  and  well  as  general  and  .specific 
descriptions  of  our  products  and  services.  We  provided  more  in-depth  looks  at  .specific 
technologies  in  the  field  such  as  process  automation  u.sing  modem  .software  tools  and  methods. 

Selected  Papers/Proceedings  describing  software  engineering  conferences  or  activities 
supported  by  the  DACS  or  DACS  staff  members  are  periodically  published  to  capture  the 
activities  of  the  conference,  sympo.sia,  or  meeting.  Additionally,  papers  on  aspects  of  software 
engineering  were  prepared  for  publication  by  members  of  the  DACS  staff  for  pre.sentation  at 
conferences  or  publication  in  appropriate  journals. 
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Newsletters/Special  Bulletins  wcr‘  plJbll^hcd  by  the  DAC'S  ihrouehoui  ihc  vcar.  NcwNlciiors 
were  published  quarterly  and  bulletins  are  published  eight  times  per  year  tu  eoineide  with  the 
months  a  newsletter  was  not  published.  During  this  eontraet  the  newsletter  has  been 
substantially  expanded  and  the  quality,  format,  and  eontent  updated.  The  artieies  and 
information  in  the  newsletter  were  geared  to  meet  the  needs  and  desires  of  our  readers.  The 
surveys  we  have  condueted  in  the  past  have  sugge.sted  the  sort  of  mix  of  news,  calendars  and 
lechnieal  content  that  we  provide.  Numerous  favorable  comments  ha\e  been  received  on  the 
newsletter  from  our  user  community.  The  DAC.S  bulletin  has  a  much  more  limited  distnbution 
and  provides  information  predominately  to  our  local  Rtime  Laboratory  community.  A  popular 
area  of  coverage  has  been  conferences  on  a.specLs  of  software  technology  and  DACS  activities  in 
the  field. 

Specialized  Software  Data  Repositories  are  maintained  at  the  IdACS.  We  have  successfully 
arranged  to  hou.se  data  from  members  of  the  American  Insiituie  for  Aeronautics  and  A.stroi  autic.s 
(AIAA)  at  the  DACS.  Several  dala.sets  have  been  received  for  test  and  evaluation  purposed.  We 
ahso  arranged  to  receive  Space  Command  Data  forevaluation  and  analysis.  As  a  part  of  a  special 
study,  we  accepted  the  Robotics  and  Artificial  Intelligence  Database  (RAID)  and  have  integrated 
it  into  our  holdings.  It  is  anticipated  that  with  the  lack  of  special  task  funding,  the  DAC'S  will 
augment  RAID  operation  into  a  CORE  activity. 

Siniulation/Model  Development  &  Evaluation  was  periodically  provided  to  our  users  by 
DACS  staff  members.  The  data  is  often  reliability  or  cicsi  data  and  models  such  as  the  Goel- 
Okumoto  model  are  u.sed  for  analysis  and  reporting  purposes.  As  a  part  of  a  technical  area  task 
for  the  U.S.  Army,  a  software  metrics  analysis  tool  is  being  proioiv  ped  for  use  by  Army  program 
managers.  It  will  be  distributed  and  supported  at  the  DACS  in  future  activities  with  the  Army 
software  engineering  community  and  it  may  be  augmented  into  the  DoD  commundy  at  a  future 
date.  We  have  also  been  involved  in  model/framewvirk  development  and  testing  for  aspects  of 
.software  quality  and  .software  reu.se  component  certification. 

Our  free  informational  guides  to  DACS  Products  and  Services  described  our  traditional  state-of- 
the-art  reports  and  critical  review.s/technology  as.se.ssment.s,  and  our  daiaba.se  products.  The 
following  pages  are  a  repre.sentation  of  our  DAC.S  Products  &  Services  guide  describing  CORE 
support  to  the  defen.se,  academic  and  commercial  DAC.S  u.sers. 


CURRENT  AWARENESS  PUBLICATIONS 

•  DACS  NEWSLETTER.  The  DACS  Newsletler  is  a  quarterly  puolication  (Marcn,  June.  Seoiemder.  and 
Decemberi  that  provides  readers  wi(h  a  general  awareness  ol  signiiicanl  developments,  trends,  and  technical 
activities  in  the  software  field. 

INFORMATION  PACKAGES 

•  ADA  COMPILATION  SYSTEM  (ACS)  INFORMATION  PACKAGE.  PacKage  contains  ordering  information  and 
a  'Statement  of  Terms  and  Conoitions'  for  tne  Air  Forces  ACS  software  and  documentation 

•  ADA  COMPILER  EVALUATION  CAPABILITY  (ACEC)  INFORMATION  PACKAGE.  Package  contains  ordering 
information  and  a  'Statement  of  Terms  and  Conditions'  for  the  Ada  Compiler  Evaluation  Capability  (ACEC). 
software  and  documentation 

«  ADA  FEATURES  IDENTIFICATION  SYSTEM  (AFIS)  INFORMATION  PACKAGE.  Package  contains  ordering 
information  and  a  "Statement  of  Terms  and  Conditions'  for  the  Ada  Features  Identification  System  (APIS) 
software  and  documentation. 

.  AIR  FORCE  ARMAMENT  LABORATORY  (AFATL)  ADA  COMPILER  INFORMATION  PACKAGE.  Package 
contains  ordering  information  and  a  ‘Stafement  of  Terms  and  Conditions’  for  the  Air  Forces  AFATL  Ada 
Compiler  and  Interactive  Debugger. 

■  COMMON  ADA  MISSILE  PACKAGES  (CAMP)  INFORMATION  PACKAGE.  PacKage  contains  ordering 
information  and  a  'Statement  of  Terms  and  Conditions'  for  the  CAMP  products 

•  DACS  INFORMATION  PACKAGE.  A  oacKage  o(  information  describing  the  products  and  services  offered  by 
the  DACS  The  pacnage  contains  sample  DACS  Newsletters,  a  Users  Guide  ro  DACS  Products  and  Services 
iOACSGUIDE).  a  User  s  Guide  to  Bibhograpnic  Services  iBIBGUlDE).  and  fliers  on  selected  DACS  products. 

>  INTERSYSTEM  ELECTROMAGNETIC  COMPATIBILITY  ANALYSIS  PROGRAM  (lEMCAP)  INFORMATION 
PACKAGE,  Package  contains  ordering  mtormation  and  a  'Statement  ol  Terms  and  Conditions'  for  lEMCAP 
and  related  programs 

.  ROBOTICS  AND  ARTIFICIAL  INTELLIGENCE  DATABASE  (RAID)  INFORMATION  PACKAGE.  Package 
contains  an  overview  of  RAID,  forms  tor  adding  your  pro|ect.  and  instructions  for  accessing  the  database. 

.  USER  S  GUIDE  TO  DACS  PRODUCTS  AND  SERVICES  (DACSGUIDE).  A  descriptive  brochure  which 
provides  an  introduction  to  and  ordering  guidelines  for  DACS  products  and  services 

DOCUMENTS 

•  AIAA  SOFTWARE  TOOLS  SURVEY.  Raw  data  compiled  by  the  American  Institute  of  Aeronautics  & 
Astronautics  (AIAA)  Technical  Committee  on  Computer  Systems  from  a  1979  survey  of  aerospace-related 
ndustry,  government,  and  university  sottwaie  tools  m  use  or  under  development  711  pages  in  2  volumes. 
1980  LIMITED  STOCK 

«  ARTIFICIAL  NEURAL  NETWORKS  TECHNOLOGY.  Describes  wnat  artificial  neural  networks  are.  how  to  use 
■r^em.  and  where  they  are  currently  oemg  applied  Seminal  neural  network  architectures  for  prediction, 
classification,  data  association,  data  conceptualization,  and  data  filtering  are  presented  83  pages.  August  20. 
'992 

•  DACS  GLOSSARY.  Contains  definitions  of  terms  from  the  software  engineering  literature  Prepared  in 
conjunction  with  DACS  participation  on  the  IEEE  Computer  Society.  Technical  Committee  on  Software 
E.ngineerinq.  Subcommittee  on  Software  Engineering  Standards,  m  writing  the  IEEE  Standard  Software 
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Engineering  Termmoioay  ’  oages.  Marcn  1981 

•  A  DESCRIPTIVE  EVALUATION  OF  SOFTWARE  SIZING  MODELS.  P  'Ovices  a  comprenensive  'eview  ana 
:ritiQue  oi  software  s^z  na  •"ooeis  oaseo  on  Source  L^nes  or  Coce  mat  are  awanaD  e  to  Air  Force  Cost  "Centers 
The  reoon  uluslraies  general  aoproacnes  to  sizing,  provioes  a  consistent  set  ot  information  tor  eacn  automateo 
mooei.  loentifies  aata  reouireo  to  aopiy  tne  mooeis.  cianiies  output  results,  ana  rates  eacn  mooei  accoraing  to 
ts  relevance  to  mtenoeo  users  '67  pages.  Seoiemoer  '987 

•  A  DIRECTORY  OF  AIR  FORCE  ADA  &  JOVIAL  SOFTWARE  ENGINEERING  TOOLS.  A  usting  o'  software 
engineering  toois  useo  by  United  Stales  Air  Fo'ce  organizations  and  meir  contractors  to  deveop  and  support 
software  systems  implemented  in  me  Ada  and  Jovial  programming  languages  Based  on  a  DACS  database  and 
a  survey  conducted  in  late  '986  ano  1987  257  pages.  February  1988 

•  DOD-STD-2167A,  SOFTWARE  DEVELOPMENT  STANDARD  (FEB  1988).  The  loml  military  standard  tor 
Defense  System  Software  Development  ano  the  supporting  Data  Item  Descriptions  (DIDS).  Available  either  in 
hardcopy  or  on  floppy  disk.  DOD  STD'2168.  Defense  System  Soflware  Quality  Program:  MIL-ST0-483A. 
Configuration  Management  Practices  tor  Systems.  Eguipmenl  Munitions,  ana  Computer  Programs:  MIL-STD- 
490A.  Specification  Practices:  and  MIL-STD-1521B.  Technical  Reviews  ana  Audits  for  Systems.  Equipment:, 
and  Computer  Programs  are  bundled  with  the  hardcopy  version  The  floppy  disks  contain  ASCII  files  for  DOD- 
STD-21 67A  ano  associated  DIDs. 

•  EVALUATION  OF  ACEC  ANO  PIWG  BENCHMARK  SUITES  FOR  ADA.  This  report  evaluates  two  benchmark 
suites  for  Ada.  These  suites  were  analyzed  with  respect  to  evaluating  compilers  tor  real-time  embedded 
applications  15  pages.  April  1989. 

•  EVALUATION  OF  UNIVERSITY  OF  MICHIGAN  BENCHMARK  SUITES  FOR  ADA.  This  repon  evaluates  the 

University  of  Michigan  oencnmarK  suite  for  Ada  The  suite  was  analyzed  with  respect  to  evaluating  compilers  tor 
real-time  emoeooeo  applications  13  pages.  April  1989 

•  GLOSSARY  OF  SOFTWARE  QUALITY  TERMS  (DRAFT).  This  orah  glossary  provides  a  single  reference  for 
definitions  of  terminology  related  to  software  quaniy  The  terms  are  derived  from  effons  related  to  software 
quality  sponsored  by  Rome  Laboratory,  the  National  Bureau  ot  Standards,  ano  the  Air  Force  Operational  Test 
and  Evaluation  Center  18  pages.  November  t985. 

•  AN  OVERVIEW  OF  OBJECT  ORIENTED  DESIGN.  This  reports  summarizes  the  history  of  OOD,  presents  an 
OOD  methodology,  evaluates  programming  language  suopog  for  OOD,  and  explores  OOD's  impact  on  software 
development  83  pages.  April  1991. 

•  PIWG  ADA  PERFORMANCE  BENCHMARKS;  EXECUTION  RESULTS.  This  report  documents  the  results 
from  running  me  ACM  SIGAda  Performance  Issues  Working  Group  (PIWGi  benchmarks  on  a  Motorola  68020 
computer  with  a  MC  68881  math  co-processor  using  the  Verdix  SUN/UNIX  cross-compiler  and  a  MicroVAX  II 
using  a  self-hosted  DDC-I  compiler  27  pages.  April  1989. 

.  PROCEDURES  FOR  COMPUTER-AIDED  SOFTWARE  ENGINEERING  TOOL  ASSESSMENT.  Describes 
procedures  tor  life  cycle  assessment  of  CASE  toois  lor  tactical  embedded  systems.  The  procedures  were 
developed  by  a  DACS  study  66  pages.  April  1989 

•  QUANTITATIVE  SOFTWARE  MODELS.  Describes  software  litecyae  cost  and  productivity,  software  reliability, 
and  software  complexity  mooeis  ana  methods  Includes  matrices  of  data  parameters  for  each  method  (See 
3iso  Software  Measurement  Models  )  '59  pages.  March  1979  LIMITED  STOCK 

.  REAL-TIME  ADA  PERFORMANCE  BENCHMARKS;  EXECUTION  RESULTS.  This  report  documents  the 
results  from  running  the  Real-time  Ada  Performance  Benchmarks  on  a  Intel  80386  computer  using  the  Verdix 
DDC-I  Ada  compiler  at  pages.  April  1989. 

.  A  REVIEW  OF  SOFTWARE  MAINTENANCE  TECHNOLOGY.  Information  on  software  maintenance  tools  and 
techniques  compiled  m  1979.  This  report  contains  matrices  relating  each  tool  or  technique  to  various 
maintenance  functions,  an  extensive  bibliography,  and  a  glossary  of  terms.  221  pages,  February  1980.  LIMITED 
STOCK. 

.  SOFTWARE  ANALYSIS  AND  TEST  TECHNOLOGIES.  Examines  current  software  analysis  and  test 
technology  and  needs  that  should  be  filled  by  future  technology.  Analysis  and  testing  includes  all  life  cycle 
activities  conducted  to  verify  and  validate  the  software  product  80  pages.  February  1992. 

•  SOFTWARE  LIFE  CYCLE  TOOLS  DIRECTORY.  Contains  information  about  4t2  software  life  cycle  tools  Also 
contains  a  listing  of  tool  acronyms  by  tool  classification,  implementation  language,  target  computer,  operating 
system,  and  features  for  easy  cross  referencing  472  pages.  March  1985 

•  SOFTWARE  MEASUREMENT  MODELS.  A  summary  of  software  engineering  measurement  models  This 
report  includes  information  on  software  reliability  models,  life  cycle  cost  models,  software  sizing  models  and 
software  complexity  metric  models.  (Updates  Quantitative  Software  Models  )  156  pages.  July  1987 

•  SOFTWARE  PROTOTYPING  ANO  REQUIREMENTS  ENGINEERING.  Describes  the  motivation  for  software 
prototyping  in  general  and  specifically  m  requirements  engineering  Summary  analyses  of  software  requirements 
and  specification  techniques  and  prototyping  tools  cover  20  techniques  and  tools  162  pages.  June  1992 
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>  SOFTWARE  RELATED  PROBLEMS  AT  THE  CENTER  FOR  SOFTWARE  ENGINEERING.  Assesses  sotiwate 
Biaieo  Dfooiems  encouniereo  me  aay  to  day  ooetanons  at  me  Ceniet  tor  Sonwate  Engineefng  \CSEl  at 
CECOM.  US  Army.  Ei  Monmoutn,  NY  Snows  now  CASE  toO'S  couio  neiD  CSE  personnel  n  aodressing  me 
ceniitiea  orooierns  ’7  oages.  Aoni  1989 

•  SOFTWARE  REUSABILITY.  A  summary  oi  the  stale  oi  me  an  .n  sonware  'eusaomiy  T^is  report  descnoes 
mportant  reusaoilily  proiects  around  me  wond  Ada  repositories  m  tne  U  S  ana  prooiem  areas  mat  hinder 
reusaoimy  from  oeinq  common  practice.  100  pages.  August  1990 

•  STARS  MEASUREMENT  SURVEY  SUMMARY.  A  summary  ot  me  results  oi  a  survey  to  dentily  existing 
measurement  dataoases.  models,  and  software  tools  conducted  from  August  ’985  to  Oecemoer  1985.  i62 
pages.  May  ’Q86 

.  A  STATE  OF  THE  ART  REVIEW  OF  DISTRIBUTED  DATABASE  TECHNOLOGY.  Reviews  me  issues  that 
arise  with  Disiriouted  OBMSes.  surveys  commercially  availaoie  Distnouteo  OBMSes.  and  summarises  the  state 
ot  the  art  dO  pages.  Ocfooer  1992 


□  SOFTWARE  ENGINEERING  DATASETS  □ 


DATASETS 

.  ARCHITECTURE  RESEARCH  FACILITY  (ARF)  ERROR  DATASET.  Data  descnoes  n7  error  reoons. 
software  characteristics  data  on  253  modules,  and  protect  aggregates  tor  mo  ARF  oeveioood  at  me  Naval 
Research  Laboratory  m  the  late  1970s  Avaiiaole  on  an  MS-DOS  tioopy  disk  or  .n  naracooy  lorm 

•  OACS  PRODUCTIVITY  DATASET.  This  dataset  contains  summary  information  'rom  over  500  sonware 
protects,  incorporating  size  data,  error  data  proiect  duration  toiai  etlod.  language  data,  and  mlormanon  on  me 
usage  of  various  software  implementation  technologies  iSee  Software  Data  CoHechon  ana  Analysis.)  Availaoie 
on  MS-DOS  floppy  disK 

•  NASA/AMES  F.RROR/FAULT  DATASET.  Error/Fault  data  on  3700  sonware  prooiem  reports  collected  on  nine 
projects.  Data  was  originally  compiled  by  NASA/Ames  Research  Center  m  tne  late  i970s  Available  on  an 
ANSI  standard  9  track  tape  or  in  hardcopy  form, 

•  NASA/SEL  DATASET.  Data  collected  by  the  Software  Engineering  Laboratory  (SEL).  at  NASA  Goddard  Flight 
Center,  to  measure  the  effectiveness  of  software  development  methodologies  The  dataset  contains  over 
45,000  records;  the  ma|orily  ot  the  dataset  is  from  component  status  repons  and  run  analysis  reports  The 
remainder  of  the  dataset  is  project  comment  information,  change  reports,  resource  summary  reports,  and 
component  summary  reports.  Last  updated  in  December  199t,  Available  on  3  ANSI  standard  9  track  tapes 

•  PAVE  PAWS  OPERATIONS  AND  MAINTENANCE  (OAM)  DATASET.  04M  data  coilecteo  on  the  PAVE 

Phased  Array  Warning  Systems  (PAWS)  in  the  late  1970s  and  eariy  i98Cs  Available  on  an  ANSI  standard 
tape. 

•  SOFTWARE  RELIABILITY  DATASET.  Failure  data  on  16  software  systems  collected  during  the  phases  ot 
software  lest  and  operation  during  me  1970s  Suitable  for  validating  software  'enaonity  models  iSee  Software 
Reliability  Data,  l  Available  on  an  MS-DOS  floppy  disk 

.  VALIDATION  AND  VERIFICATION  (VSV)  DATASET.  Data  summarized  f'om  '  500  anomaly  reports  on  '.ve 
VSV  projects  during  the  late  1970s  Available  on  an  ANSI  standard  tape  oi  as  a  narocooy  I'Stmg 

RELATED  PRODUCTS 

•  A  COMPARISON  OF  DACS  AND  NASA/SEL  SOFTWARE  DEVELOPMENT  DATA.  This  report  presents  a 
statistical  analysis  of  productivity  data  from  the  NASA/SEL  and  DACS  Productivity  Datasets  27  pages. 
December  1982. 

•  OACS  DATA  COMPENDIUM.  A  description  of  software  experience  data  available  from  the  DACS  This 
document  includes  type  of  data,  number  of  records  of  each  type  ecoro  tormats,  and  instructions  for  obtaining 
data.  66  oages.  April  1992. 

.  THE  DACS  SOFTWARE  ENGINEERING  DATA  COLLECTION  PACKAGE.  Produced  by  the  DACS  as  pah  of 
an  effoh  to  organize  the  data  items  necessary  to  suppoh  analysis  activities  mio  a  classification  scheme  and  to 
promote  standardized  collection  of  software  engineering  data  The  document  contains  an  overview  of  the  data 
collection  process,  a  classification  scheme  for  the  different  types  of  data  wnich  may  be  collected,  examples  lor 
applying  the  package  to  specific  types  of  analyses,  data  collection  forms  and  instructions  for  completing  the 
forms  a  glossary  of  terms  and  data  items,  and  an  evaluation  questionnaire  87  pages.  March  1984 

•  NASA/SEL  DATA  COLLECTION  FORMS.  Forms  used  by  the  NASA  Software  Engineering  Laboratory  (SEL) 
to  collect  life  cycle  data  on  programs  developed  at  the  NASA  Goddard  Space  Flight  Center.  The  forms  are 
used  to  collect  data  on  general  project  information,  changes,  resources,  components,  ano  proioct  personnel  29 
pages,  February  1977 
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•  NASA.SEL  DATA  COMPENDIUM.  'Soecilic  .r’o'f^aiion  on  29  OfOiecis  m  me  NASA  SEL  aalasei  wxiin  ooloniiai 
JODiicaliOPs  'Or  me  oaia  Contains  ’26  oages  ot  ’eit.  crarts  gtaons.  ana  to'ms  Aoni  ’  98' 

•  SOFTWARE  DATA  COLLECTION  AND  ANALYSIS  (DRAFT).  An  anaiys  s  ot  data  ''om  me  DACS"  P'oauct  v  rv 
Catasel-  rms  atati  oania,  reoort  cxmiams  statistics  scmmanzing  tna  aistnoulions  ot  ivey  vanaoies  (Size,  etton 
scPeOuie  ano  tact  censity;  a.no  'egressions  A  special  eMori  s  maoe  to  dspiay  me  eltecis  ot  mooern 
programming  practices  Septemoer  '978 

•  SOFTWARE  RELIABILITY  DATA.  Wnnen  pv  Dr  Jonn  Musa  ot  Bell  Telepnone  LaDoratorios,  mis  report 
summarizes  me  soitkvare  'enaOilily  aataset  ano  tits  tne  Musa  e«ecuIion-iime  mooei  to  the  oaia.  1  73  pages 
January  1980 

.  STARS  INTERIM  SOFTWARE  ENGINEERING  DATA  COLLECTION  FORMS  SET  A  set  ot  e-gnt  documents 
Executn/e  Overview  and  Fmai  Reoort  on  the  interim  Sollware  Data  CoDeclion  Forms  Deveiopmenl  and  six 
Interim  Sotiwate  Data  Collection  Forms  covering  Resource  Expenditure.  Software  Ctiaractenstics.  Software  Test 
Information.  Software  Prooiem/Cnange,  Software  Environment,  ano  Software  Evaluation  327  pages.  June  t965 


□  SOFTWARE  ENGINEERING  BIBLIOGRAPHIC  AND  PROJECTS  DA TABASE  □ 


The  Software  Engineering  Bibliograpnic  DataOase  (SEBD)  is  collection  ot  citations  and  abstracts  tor  over  13,000 

software  engineering  ana  software  technology  texts,  technical  reports,  theses,  lournal  articles,  proceedings, 

standards,  ano  otnet  documents  fhal  are  mainiamoa  in  the  DACS  Software  Engineering  Library. 

The  Robotics  and  Artificial  Intelligence  Database  iRAIO)  contains  contact  ano  oroiect  information  on  almost 

2000  proiecls  ano  over  9C0  organizations  involved  m  Defense-sponsored  research  in  roDolics  ano  Al 

CUSTOM  SEARCHES 

•  BIBLIOGRAPHIC  SEARCH.  Custom  search  ot  me  Software  Engineering  Bibliographic  Database  (SEBO)  Use 
the  BIBGUIDE  to  structure  a  search  ot  the  SEBD  on  one  or  more  specific  topics 

•  RAID  SEARCH.  Custom  electronic  search  of  the  Robotics  and  Artificial  Intelligence  Database  (RAID).  Use  the 
RAID  intormation  oacKage  to  structure  a  search  of  RAID.  Note:  Export-Controlled  Technical  Data.  Requirea 
sponsor  letter  tor  nongovernment  organizations. 

•  USER  S  GUIDE  TO  BIBLIOGRAPHIC  SERVICES  (BIBGUIDE).  This  document  is  a  guide  for  ordering  a  DACS 
custom  Dioiiographic  search  The  guide  contains  the  DACS  Software  Engineering  Thesaurus  of  keywords  used 
lor  indexing  and  retrieving  documents  from  me  Software  Engineering  Bibliographic  Database  (SEBD). 

BIBLIOGRAPHIES 

•  THE  DACS  ANNOTATED  BIBLIOGRAPHY.  A  bibliography  of  software  engineering  literature  contained  in  the 
DACS  Software  Engineering  Bibliographic  Database  (SEBD).  The  Bibliography  includes  citations  and  abstracts 
of  documents  m  the  DACS  Software  Engineering  Library,  a  keyword  index  built  from  the  DACS  Software 
Engineering  Thesaurus,  a  suoiect  index,  an  author  index,  and  a  Keyword-m  Context  (KWIC)  index.  Volume  I, 
'Accession  Numoers  l-f62Ai.  pupiished  August  1980:  Volume  II.  (Accession  Numbers  1625-2214),  published 
January  '982:  Volume  ill,  ; Accession  Numoers  2215-3013).  published  January  1983;  Volume  IV,  (Accession 
Numoers  3014-4513),  puoiisnea  June  1984;  Volume  V.  (Accession  Numbers  4514-5513),  published  December 
'985;  Volume  VI.  (Accession  Numoers  5514-6513),  published  February  1988;  Volume  VII,  (Accession  Numbers 
6514-7500).  puDlisheo  April  '989;  Volume  VIII,  (Accession  Numbers  7501-9000  ).  published  December  1990. 

.  THE  DACS  MEASUREMENT  ANNOTATED  BIBLIOGRAPHY.  A  bibliography  of  660  software  measurement 
documents  including  texts,  tecnnical  reports,  theses,  (ournal  articles,  conference  proceedings,  and  standards. 
The  topics  covered  m  me  oibiiograohy  include  complexity  measurement,  cost  estimation,  life  cycle  costs, 
maintenance  costs,  produclivily  data,  costing  techniques,  data  analysis,  software  experience  data,  quality 
metrics,  reliability  measurement,  reliability  models,  cost/produciivity  models,  data  collection,  and  data 
repositories.  263  pages.  May  1986. 

.  SOFTWARE  ENGINEERING  INSTITUTE  S  TECHNICAL  REPORTS.  The  Software  Engineering  institute  (SEl) 
IS  a  federally  funded  research  and  develoomeni  center  sponsored  by  the  Department  ot  Defense  (DoD)  under 
contract  to  Carnegie  Mellon  University  This  bibliography  contains  citations  of  all  SEl  documents  in  the  SEBD  as 
of  Its  production  date  41  pages,  1988 


□  PROCEEDINGS  □ 


.  KNOWLEDGE-BASED  SOFTWARE  ASSISTANT/ENVIRONMENT.  The  Knowledge-Based  Software 
Environment  (KBSE),  formerly  the  Knowledge-Based  Software  Assistant  (KBSA).  is  a  tool  for  software 
develoomeni  ano  maintenance  intended  lo  support  a  transformational  lifecvde  paradigm  The  KBSE  uses 
Artificial  Intelligence  techniques  and  guidance  to  automatically  transform  formal  requirements  to  designs  and 
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cexia.  with  maintenance  being  performed  on  the  format  requirements.  Since  t986.  Rome  Laboratory  has 
sponsored  a  conference  to  provide  technicat  interchange  between  researchers  and  developers  in  the  KBSE 
community.  Annual  proceedings  for  these  conferences  are  available  through  the  OACS  for  1986  through  1992. 

•  SOFTWARE  QUALITY  WORKSHOP.  Four  the  last  lour  years.  Roma  Laboratory  has  sponsored  an  annual 
workshop  on  software  quality.  Attendees  and  presenters  have  discussed  all  aspects  of  software  quality  and 
software  quality  measurement.  Annual  proceedings  are  available  from  the  DACS  lor  the  first  tour  annual 
workshops,  from  1 989  to  1 992. 


□  SOFTWARE  ENGINEERING  TOOLS  □ 


•  ADA  COMPILATION  SYSTEM  (ACS).  The  Air  Force's  400,000-line  fully  self-compiled  Ada  compiler  runs 
under  the  UTS  operating  system  on  IBM  370.  43XX,  and  30XX  computers.  The  UTS  operating  system  is  a 
Unix  variant  available  from  Amdahl  either  in  "native  mode"  lor  Amdahl  580-series  machines  or  on  top  of  VM  on 
IBM  machines.  This  compiler  was  developed  for  Rome  Laboratory  by  Intermetrics,  Inc.,  and  is  distributed  on 
two  9-track.  6250  bpi,  Unix  tar  tapes.  Note:  Export-Controlled  Technical  Data.  Raqulroa  eomplatad 
‘Slatament  of  Tarms  and  Conditions*  for  distribution  to  any  organizations  other  than  those  within  the 
Ak  Force. 

•  ADA  COMPILER  EVALUATION  CAPABIUTY  (ACEC).  The  Ada  Compiler  Evaluation  Capability  (ACEC)  was 
developed  by  Boeing  Military  Airplane  Company  for  the  Air  Force  Wright  Aeronautical  Laboratories.  Its  purpose 
is  to  provide  the  capability  to  determine  the  performance  characteristics  of  Ada  compilation  systems.  The 
ACEC  products  include  the  ACEC  Software  Product  and  three  supporting  documents:  the  ACEC  User's  Guide, 
the  ACEC  Version  Oescripiian  Document  (VOO).  and  the  ACEC  Reader's  Guide.  The  ACEC  Software  Product 
consists  of  operational  software  and  support  software.  The  ACEC  Software  Product  was  developed  for 
uniprocessor,  uni-programming  target  systems  and  is  distributed  on  one  9-track,  1 600  bpi.  VAX/VMS  "backup  ' 
tape.  Requires  completed  ‘Statement  of  Terms  and  Conditions’  lor  distribution  to  any  organizations 
other  than  those  within  the  Air  Force. 

•  ADA  FEATURES  IDENTIFICATION  SYSTEM  (AFIS).  The  Ada  Features  Identification  System  (APIS)  was 
developed  by  New  York  University  for  the  ACVC  Maintenance  Organization  (AMO)  at  Wright-Patterson  Air 
Force  Base.  Its  primary  purpose  is  to  aid  the  evaluation  and  maintenance  of  the  Ada  Compiler  Validation 
Capability  (ACVC),  but  it  can  be  used  to  determine  what  features  are  present  in  any  Ada  program.  The  AFIS 
consists  of  the  AFIS  software  product,  the  AFIS  User's  Manual.  An  Introduction  to  the  PAT  Input  Language,  and 
the  PAT  Language  Reference  Manual.  AFIS  is  distributed  either  on  three  9  track,  1600  bpi  Unix  tar  tapes  or  on 
seven  9  track.  1600  bpi  ANSI  standard  tapes.  Requires  completed  ‘Statement  of  Terms  and  Conditions’ 
for  distribution  to  any  organizations  other  than  those  wHhln  the  Air  Force. 

•  AIR  FORCE  ARMAMENT  LABORATORY  (AFATL)  ADA  COMPILER  AND  ADA  INTERACTIVE  DEBUGGER. 

The  Air  Force's  AFATL  compiler,  written  in  Pascal,  hosted  on  the  CYBER  176,  and  targeted  to  the  ZS002 
microprocessor,  was  validated  in  October  1985.  The  debugger  is  written  in  Pascal  and  runs  on  the  VAX  11/780. 
The  AFATL  Ada  Compiler  and  Interactive  D^ugger  are  distributed  on  two  9-track  ANSI  standard  tapes.  Note: 
Export-Controlled  Technical  Data.  Requires  completed  ‘Statement  of  Terms  and  Conditions*  for 
distribution  to  any  organizations  other  than  those  within  the  Air  Force. 

•  COMMON  ADA  MISSILE  PACKAGES  (CAMP).  The  CAMP  products  include  CAMP  Parts,  the  CAMP 
Armonics  Benchmarks,  and  the  CAMP  Parts  Engineering  System  (PES).  CAMP  Parts  are  444  reusable  Ada 
components  organized  into  35  Top-Level  Computer  Software  Components  (TLCSCs)  which  contain  137,000 
source  lines  of  Ada  code.  The  CAMP  Armonics  Benchmarks  are  used  to  evaluate  Ada  and  processor 
implementations  in  the  armonics  domain.  The  tests  establish  the  correctness  of  compiler  implementations  and 
measure  performance  in  size  and  speed  of  generated  code.  The  CAMP  PES  provides  mechanisms  for 
identifying  and  retrieving  reusable  software  parts,  adding  new  parts  to  the  catalog,  and  data  administrator 
functions.  The  PES  runs  on  VAX  VMS  systems.  CAMP  Parts  are  distnbuted  on  two  ANSI  standard  labeled  9- 
track  1600  bpi  tapes.  The  Armonics  Benchmarks  are  distributed  on  one  9-track  1600  bpi  ANSI  standard  tape. 
The  CAMP  PES  are  distributed  on  three  9-track  ANSI  Standard  tapes,  two  of  which  are  6250  bpi  and  the 
remainder  is  1600  bpi.  Requires  completed  ‘Statement  of  Terms  and  Conditions’  for  distribution  to  any 
organizations  other  than  those  within  the  Air  Force. 

•  GOEL-OKUMOTO  SOFTWARE  REUABIUTY  MODEL  An  automated  version  of  the  Goel-Okumoto 
Nonhomogeneous  Poisson  Process  Software  Reliability  Model  which  runs  on  an  IBM-PC  or  compatible  under 
MS-DOS  2.11  or  higher.  Features  include  the  ability  to  find  maximum  likelihood  estimators  of  the  parameters 
defining  the  model  by  using  either  the  Newton-'Raphson  or  Muller's  method;  a  goodness-of-fit  test  based  on  a 
Kolmogorov-Smirnov  statistic:  estimation  of  remaining  faults,  cumulative  failures,  and  reliability;  and  estimation 
of  the  optimal  release  time  based  on  certain  cost  criteria.  This  program  is  distributed  on  5  1/4"  MS-DOS  floppy 
disk. 

•  INTERSYSTEM  ELECTROMAGNETIC  COMPATIBILITY  ANALYSIS  PROGRAM  (lEMCAP).  The  DACS 
distributes  certain  Air  Force  Electromagnetic  Analysis  Codes  of  which  the  leading  one  is  lEMCAP.  Requires 
completed  ‘Statement  of  Terms  and  Conditions’  for  distribution  to  any  organizatlona  other  than  those 
within  the  Air  Force. 
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z:  PRODUCTS^  SERVICES  ORDERING  INFORMATION  □ 

PARTICIPATION  PLAN 

The  DACS  Full  Service  Participation  Plan  provioes  lull  access  to  DACS  resources  ihrougn  payment  o(  a  single 
participation  fee  Participants  can  request  consultations  custom  software  loot  searcnes.  custom  oibiiographtc 
searcnes,  ano  DACS  puolications  Participants  receive  one  copy  of  each  puoiicaiion  when  issued,  discount 
privileges,  access  to  DACS  resources,  ana  account  maintenance  A  DACS  Full  Service  Participation  Plan  can  be 
opened  by  depositing  a  minimum  of  $500  (the  maximum  to  bo  determined  by  the  requestor)  The  DACS  will  send 
the  user  a  summary  of  plan  purchases  on  a  quarterly  basis  Contact  the  DACS  to  establish  a  Full  Service 
Participation  Plan 

EXPORT-CONTROLLED  MATERIAL 

As  a  OoD  Information  Analysis  Center,  the  DACS  is  authorized  to  distribute  unclassified  export-oontrolled 
technical  data,  publications,  and  information.  The  DOD  Directive  5230.25  tDD5230.25).  "WithhoWing  of 
Unclassified  Technical  Data  from  Public  Disclosure.'  limits  the  distribution  of  unclassified  export-controlled  techncal 
data  to  organizations  certified  as  qualified  contractors  by  the  Defense  Logistics  Services  Center  (DLSC)  Qualified 
contractors  appear  on  the  OLSC's  Qualified  Contractor  Access  List  fOCAL)  Non-Government  organizations  can 
apply  for  DLSC  certification  by  submitting  a  certification  applicaiion  iDD  Form  2345,  April  1906;  Militarily  Critical 
Technical  Data  Agreement)  to  the  DLSC  (Note;  Government  activities  oo  not  have  to  submit  to  certification.) 

STATEMENT  OF  TERMS  AND  CONDITIONS 

To  obtain  Air  Force-owneq  products  distributed  by  me  DACS.  a  Statement  or  Terms  and  Conditions;  Release 
of  Air  Force-Owned  or  Developed  Computer  Software'  must  be  completed  by  ail  non-Air  Force  organizations.  The 
Statement  must  be  approved  by  the  U  S  Air  Force  before  any  products  can  be  shipped  to  an  organization  by  the 
DACS. 

FOREIGN  ORDERS 

The  United  States  Department  of  Defense  only  allows  the  expon/transfer  of  products  originated  by  the  Data  and 
Analysis  Canter  for  Software  to  foreign  requesters  on  an  international  government  to  government  basis.  A  written 
request  should  be  sent  to  the  appropriate  foreign  country  s  embassy  identifying  the  DACS  as  the  source  of  these 
technical  reports  The  embassy  snould  then  forward  the  request  to; 

U.S.  Air  Force  Systems  Command 
HQ  AFSC/XTIO 

Andrews  Air  Force  Base,  MO  20334-5000 

They  in  turn  will  send  the  request  to: 

U.S.  Air  Force  Rome  Laboratory 
RUINF 

GriHiss  AFB,  NY  13441-5700 

The  requester  will  be  notified  when  formal  approval  has  been  received 

MAGNETIC  MEDIA  FORMATS 

Diskettes:  Selected  DACS  produas  are  available  on  disKetles.  Diskettes  are  5  1/4",  double  density,  double 
sided,  MS-DOS  format 

Magnetic  Tape,  Selected  DACS  products  are  also  available  on  magnetic  tape  All  magnetic  tapes  are  9-track. 
Depending  on  the  product,  they  are  either  1600  bpi  or  6250  bpi  and  m  a  Unix  tar,  ANSI  standard,  or  VMS  backup 
format. 

METHODS  OF  PAYMENT 

Military  Agencies;  Blanket  Purchase  Agreement,  DD  Form  1155.  may  be  used  tor  ordering  DACS  products  and 
services.  Please  stipulate  the  maximum  dollar  amount  authorized,  cutoff  date,  and  specify  services  to  be  provided 
(publications,  search  services,  datasets,  etc  ).  Identify  vendor  as  Kaman  Science  Corporation,  Data  A  Analysis 
Canter  tor  Software 

Except  for  DACS  Full  Servce  Participation  Plan  and  DD  Form  1155  orders,  pre  payment  of  orders  is  required. 
Please  make  checks  payable  to  Kaman  Science  Corporation.  ’ 

For  additional  information  or  ordering  assistance  contact  the  DACS  at  (315)  734-3696.  Send  completed  order 
form  and  payment  to  the  address  on  the  next  page 


INFORMATION  PACKAGES 


!  ACS  Infoftnalion  Package  *  Fotto 


ACEC  Inlormanon  Package 

Free 

Forms  ' 

APIS  Inlormation  Package 

^ree 

Forms 

AFATL  Ada  Compner  Inlormanon  Package 

Free 

Forms  , 

CAMP  Inlormanon  Package 

Free 

Forms  11 

OACS  Intormacion  Package 

1  Free 

Package 

(EMCAP  InformaDon  Package 

Free 

Forms  j! 

RAID  Inlormanon  Package 

.  Free 

Forms  j 

Users  Guide  to  DACS  Products  &  Services  iDACSGUIOEi 

Free 

Oocumeni 

DOCUMENTS  A  RELATED  PRODUCTS 


AIAA  SotNware  Tools  Survey 

550 

(2)  Documents 

Amlcial  Neural  Networks  Tecnnoiogy 

$30 

Document 

OACS  Glossary 

$30 

Document 

A  Descnotive  Evatuatioo  ot  Sohware  Sizing  Mooeis 

$15 
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DATASET  RELATED  PRODUCTS 
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8.1  Software  Tools 


We  also  made  a  number  oi  tools  available  that  were  developed  outside  the  DACS  but  made 
available  to  our  users  as  a  service  to  the  defense  software  engineering  community.  These  tools 
were  provided  on  a  cost  reimbursement  basis. 

The  DACS  distributes  certain  software  tools  as  a  .service  to  other  Defense  organizations.  A 
software  tool  is  any  computer  program  or  collection  of  software  used  in  the  development  ol  other 
program.s.  Compilers,  debuggers,  test  suites  and  benchmarks,  libraries  of  reusable  parts,  and 
programs  implementing  models  of  .software  projects  are  all  examples  of  software  tools. 

The  DACS  distributes  the  following  tools; 

•  Ada  Compilation  System  (ACS) 

•  Ada  Compiler  Evaluation  Capability  (ACEC) 

•  Ada  Features  Identification  System  (APIS) 

•  Air  Force  Armament  Laboratory  (AFATL)  Ada  Compiler 

•  Common  Ada  Mi.ssile  Packages  (CAMP)  Products 

•  A  Computerized  Implementation  of  the  Goel-Okumolo  Software  Reliability  Model 

•  The  Software  Engineering  Cost  Model  (SECOMO) 

CAMP,  the  AFATL  Ada  Compiler,  APIS,  and  ACS  are  all  export  controlled  products. 


SOFTWARE  ENGINEERING  COST  MODEL 

The  Software  Engineering  Cost  Model  (SECOMO)  is  an  interactive  software  cost  estimation 
system  that  calculates  the  total  staffing  requirements  for  a  Life  Cycle  Software  Engineering 
(LCSE)  Center  of  the  Army  Materiel  Command  (AMC).  The  technical  direct-charged  labor  for 
developing  and  maintaining  a  system  is  ba.sed  on  the  Constructive  Co.st  Model  (CfXTOMO). 
SECOMO  also  provides  estimates  of  certain  non-sy.stem  specific  personnel,  the  division  of  effort 
between  the  government  and  contractors,  and  the  division  of  cost  among  different  types  of 
funding. 

SECOMO  outputs  Include  effort  in  man-months,  schedules  in  months,  staffing  estimates,  and 
costs  in  dollars.  Thc.se  estimates  are  decomposed  by  system,  life  cycle  pha.se.  fi.scal  year,  and  .so 
on.  Graphical  outputs  are  provided,  where  appropriate. 
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THE  GOEL-OKUMOTO  MODEL 


The  DACS  developed  a  compuierized  implemenialion  of  ihe  GiK'l-Okurm>to  Software  Reliability 
Model  for  its  own  analysis  purposes.  The  Goel-Okumiiio  model  is  one  of  many  software 
reliability  models  developed  over  the  past  two  decades  hy  .software  engineering  researchers. 
These  models  describe  how  a  software  program  fails  dunrig  either  test  or  operations.  The  Goel- 
Okumoto  model  assumes  .software  fails  as  a  Non-homogeneous  Poisson  Process,  one  of  the 
simpler  stochastic  proce.sses  with  non-con.stant  failure  rates.  A  program's  failure  rate  is  assumed 
to  be  directly  proportional  to  the  number  of  bugs  it  contains.  The  model  assumes  that  a  single 
bug  is  removed  immediately  after  every  failure.  Although  DACS  re.seareh  has  shown  this 
assumption  is  not  essential.  Dr.  Goel  and  Dr.  Okumoto  a.s.sumed  that  no  errors  are  made  while 
debugging. 

Once  the  model  has  been  fit  to  u.ser  provided  data,  the  computerized  program  can  produc  ' 
various  performance  measures.  These  measures  consi.si  of  the  expected  number  of  failures  that 
will  occur  by  any  lime,  the  remaining  number  of  bugs  in  the  software,  reliability,  and  confidence 
bounds  on  all  these  measures.  The  program  al.so  provides  a  cost  model  (or  minimizing  life  cycle 
cost  based  on  reliability  considerations. 


COMMON  ADA  MISSILE  PACKAGES 
(CAMP) 


The  DACS  distributes  three  Common  Ada  Mi.s.sile  Packages  (CAMP)  products  developed  by  the 
McDonnell  Douglas  Astronautics  Company  under  contract  to  the  Air  Force  Armament  Test 
Laboratory  (Eglin  AFB,  FL).  These  products  consi.st  of  the  CAMP  Parts,  CAMP  Armonics 
Benchmarks,  and  CAMP  Parts  Engineering  System. 

The  CAMP  Parts  are  444  reusable  Ada  components  organized  into  .'^5  Top-Level  Computer 
Software  Components  (TLCSCs)  which  contain  137,(MK)  source  lines  of  Ada  code  (including 
comments,  package  specifications,  package  bodies,  and  test  code).  CAMP  Parts  are  di.stributed 
on  two  ANSI  standard  labeled  9-track  1600  bpi  tapes.  The  CAMP  Armonics  Benchmarks  are 
used  to  evaluate  Ada  and  processor  implementations  in  the  Armonics  domain.  The  Benchmarks 
represent  typical  Armonics  applications  and  include  missile  operational  parts  as  well  as  support 
parts  from  the  mathematical  domain.  The  tests  establish  the  correctness  of  compiler 
implementations  and  measure  performance  in  size;  and  speed  of  generated  code.  The  documents 
the  CAMP  Armonics  Benchmarks  Suite  and  the  CAMP  Armonics  Benchmarks  Suite  Inventory 
are  shipped  with  the  CAMP  Armonics  Benchmarks  tape.  The  CAMP  Parts  Engineering  System 
(PES),  produced  under  the  third  phase  of  the  CAMP  product,  is  an  improved  version  of  the 
CAMP  Ada  Missile  Parts  Engineering  Expert  System  (AMPEE).  The  PES  catalog  provides  a 
means  of  identifying  and  retrieving  reusable  software  parts.  The  PES  allows  new  parts  to  be 
added  to  the  catalog.  The  PES  also  provides  for  databa.se  admini.stralor  functions,  and  even  a 
callable  interface  which  supports  the  construction  of  new  tools  on  top  of  the  PES.  Unlike 
AMPEE,  the  PES  runs  on  commonly  available  platforms,  namely  VAX  VMS  systems.  The 
CAMP  Parts  Engineering  System  Catalog  User's  Guide  is  .shipped  with  the  CAMP  Parts 
Engineering  System  tapes. 
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ADA  COMPILER  EVALUATION  CAPABILITY  (ACEC) 


The  Ada  Compiler  Evaluation  Capability  (ACEC)  was  devclopc'd  by  B(X.‘ing  Military  Airplanes 
for  the  Air  Force  Wright  Research  and  Development  Center  (WRDC).  The  DACS  currently 
distributes  release  3.0  of  the  ACEC.  The  ACEC  provides  the  capability  to  determine  the 
performance  and  u.sabilily  characteristics  of  Ada  compilation  systems.  The  ACEC  etmsists  of  the 
ACEC  Software  Product  and  three  supporting  dcKuments;  the  ACEC  User's  Guide,  the  ACEC 
Reader’s  Guide  and  the  ACEC  Version  Description  Document. 

The  ACEC  Software  Product  consists  of  performance  tests,  as.sessor  tools  and  support  software. 
The  ACEC  performance  tests  provide  a.ssistance  in  measuring  execution  time  efficiency,  code 
size  efficiency,  and  compile  time  efficiency.  The  assessor  tools  provide  assistance  in  evaluating 
symbolic  debuggers,  program  library  sy.stems,  and  compiler  diagnostics.  The  te.si  suite  does  not 
include  explicit  tests  for  existence  of  language  features. 

The  support  software  is  a  set  of  tools  and  prtKedures  which  assist  in  preparing  and  executing  the 
lest  suite,  in  extracting  data  from  the  results  of  executing  the  test  suite,  and  in  analyzing  the 
performance  measurements  obtained.  The  ACEC  Software  Product  was  developed  for  a  variety 
of  targets  and  is  distributed  on  a  nine  track  tape.  The  ACEC  documentation  is  distributed  in  both 
hard  and  soft  copy. 


AFATL  Ada  Compiler 

The  Air  Force  Armament  Laboratory  (AFATL)  compiler  and  debugger  is  a  cross  compiler 
hosted  on  a  CDC  CYBER  176.  The  compiler  generates  as.sembly  language  for  a  Zilog  ZSI)()2. 
The  compiler  was  validated  in  October  of  1985.  The  AFAT’  Ada  Compiler  is  intended  for 
laboratory  u.se  and  .should  not  be  u.sed  in  an  operational  environment  unless  further  optimi/aiion 
is  performed. 

The  compiler  is  compo.sed  of  Pascal  programs  which  support  .several  pha.ses  of  the  compilation. 
Since  the  Z8()f)2  has  no  operating  system,  a  run-time  support  environment  (RTE)  is  provided. 
The  RTE  consists  of  several  Ada  and  Z8(X)2  a.s.sembly  language  modules.  Examples  of  .some  of 
the  functions  performed  by  the  run-time  sy.stem  include  run-lime  storage  allocation,  .scheduling 
of  tasks,  run-time  checks,  and  Boating  point  arithmetic.  The  Ada  Interactive  Debugger  (AID) 
was  developed  under  contract  to  the  Air  Force  Armament  Laboratory  (AFATL)  as  a  tool  for  use 
with  the  AFATL  Ada  Compiler.  The  AID  program  is  a  menu  driven,  user  friendly,  source  level 
debugger  for  Ada  language  applications  .software.  It  is  implemented  in  Pa.scal  and  is  hosted  on 
the  CYBER  176  computer  .system.  The  AID  tool  provides  facilil'i's  to  su.spend,  monitor,  and 
trace  the  execution  of  Ada  software  by  providing  features  such  as  breakpoint  traps,  data  traces, 
and  single-step  execution.  Since  .schedule  con.straints  precluded  extensive  testing,  the  AID  tool 
should  not  be  viewed  as  production  quality  software. 
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ADA  COMPILATION  SYSTEM 


The  Air  Force's  Ada  Iniegraied  Environment  (AIE)  Ada  Compilaiion  Asiem  (AC'S)  ua.s 
developed  by  Iniei  metrics.  Incorporated  under  contract  to  Rome  Laboratoe, . 

The  4(K),(X)()-line  fully  self-compiled  Ada  compiler  runs  under  the  UTS  operaim^  s>siem  on 
IBM  370,  43XX,  30XX  computers,  as  well  as  plug-compatible  machines  .such  as  those  made  b> 
Amdahl.  The  UTS  operating  system  is  an  UNIX  variant  and  is  available  from  Amdahl. 

The  compiler  has  been  validated  with  the  Ada  Compiler  Validation  Capability  (ACVC).  version 
1.8.  It  executes  at  250  to  4(M)  lines  per  minute  on  the  IBM  4341  with  an  Ada  to  assemblN 
language  ratio  of  1  to  4.  The  compiler  contains  a  comprehensive  global  optimizer.  Other 
compiler  features  include  a  partial  implementation  of  chapter  thirteen  of  the  Ada  vuuge 
Reference  Manual  (repre.senlation  clau,ses  and  implementation-dependent  features);  extensive 
syntax  error  recovery;  compilation  statistics  gathering;  and  generation  i)l  source,  object,  and 
symbol  cross  reference  listing. 


ADA  FEATURES  IDENTIFICATION  SYSTEM 

(AFIS) 


The  Air  Force's  Ada  Features  Identification  System  (AFIS)  was  developed  by  New  York 
University  for  the  ACVC  Maintenance  Organization  (AMO)  at  Wright-Paiterson  AFB,  OH.  Its 
primary  purpose  is  to  aid  in  the  evaluation  and  maintenance  of  the  Ada  Compiler  Validation 
Capability  (ACVC)  but  it  can  be  u.sed  to  determine  what  features  are  pre.sent  in  any  Ada 
program.  The  AFIS  consists  of  the  AFIS  .software  product,  the  AFIS  User's  Manual,  A  n 
Introduction  to  the  PAT  Input  Language,  and  the  PAT  Language  Reference  Manual. 

AFIS  was  developed  as  an  aid  in  the  evaluation  and  maintenance  of  the  ACVC  test  suite.  The 
purpose  of  the  system  is  to  determine  which  of  a  set  of  specified  Ada  features  are  pre.sent  in  a 
given  Ada  program.  The  .set  of  primary  features  is  a  pre-defined,  fixed  set  of  simple  Ada 
language  features  derived  primarily  from  terms  u.sed  in  the  Ada  Language  Reference  Manual, 
particularly  the  index  and  syntax  summary.  For  the  ACVC  suite,  the  primary  features 
corresponding  to  each  test  have  been  pre-computed  and  stored  in  the  dataha.se.  Other,  non- 
ACVC  Ada  programs  can  be  compiled  and  their  .sets  of  primary  features  added  to  the  databa.se; 
thereafter  they  may  be  queried  just  like  the  ACVC  tests.  The  PAT  allows  the  u.ser  to  deal  with 
more  complex  combinations  of  features.  T3ie  u.ser  can  .specify  a  new  and  unique  combination  of 
features  by  writing  a  pattern  in  the  PAT  pattern  language.  The  AFIS  can  be  u.sed  to  determine  if 
a  given  program  or  any  program  in  the  daiaba.se  contains  a  u.ser-specified  pattern. 

Two  versions  of  AFIS  are  available  from  the  DACS.  The  UNIX  version  is  distributed  on  three. 
9-track  1600  bpi  UNIX  tar  tapes.  The  VMS  version  is  distributed  on  .seven  ANSI  .standard  9- 
track  16(K)  bpi  tapes. 
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9.0  PROMOTIONAL  ACTIVITIES 

There  is  no  specific  budget  for  promotion  of  the  DACS.  Ho';vever,  we  did  promote  the  DACS  to 
members  of  the  software  engineering  community  through  the  publication  and  distribution  of  high 
quality  textual  materials  such  as  the  DACS  Newsletter  and  a  variety  of  technical  reports 
described  elsewhere  in  this  report. 

The  DACS  staff  have  also  developed  and  maintain  a  variety  of  professional  contacts  in  the 
software  engineering,  academic  and  defense  communities  which  allow  us  to  gain  recognition  for 
professional  software  engineering  expertise  among  our  peers. 

We  have  also  sought  to  join  and  participate  with  a  variety  of  organizations  nationally  which 
provide  a  forum  for  us  to  participate  in  software  engineering  and  technology  efforts  at  a  high 
level.  Similarly  we  have  contributed  articles,  time  and  information  to  those  organizations  to 
assist  them  to  meet  their  national  and  international  goals. 

The  DACS  Director  has  contributed  directly  to  the  DoD  Software  Action  Plan,  the  Defense 
Software  Strategy  Plan,  the  US  Navy’s  Next  Generation  Computer  Re.sources  Program,  the 
DoDs  Militarily  Critical  Technology  Li.st,  and  to  In.stitute  of  Electrical  and  Electronics  Engineers 
(IEEE),  Association  for  Computing  Machinery  (ACM)  and  International  Standards  Organization 
(ISO)  standards  and  specifications  efforts.  Combined,  these  actions  have  a  positive  effect  on  the 
visibility  and  reputation  of  the  DACS  within  the  profes.sional  community. 

The  DACS  staff  has  earned  a  reputation  for  understanding  the  software  engineering  and  software 
technology  needs  of  our  users.  Through  our  participation  in  and  support  of  a  variety  of 
conferences,  symposia  and  workshops,  we  have  gained  a  substantial  amount  of  information 
which  we  have  used  to  enhance  our  internal  capabilities  and  we  have  also  made  that  information 
available  to  all  of  our  users  through  our  products  and  services. 
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lO.O  TECHNICAL  AREA  TASKS 


During  this  contract  period,  we  performed  a  wide  variety  of  special  studies.  Our  DACS  staff 
performed  special  studies  and  provided  consulting  services  to  members  of  the  defense 
community,  other  government  activities,  industry  and  academia.  The  studies  involved 
everything  from  conference  support  to  verification  and  validation  and  included  both  traditional 
and  new  customers.  Technical  and  support  areas  included  the  following: 


•  Acquisition  Support 

Distributed/Parallel  Processing 
Software  Reuse/Reengineering 
Metrics/Measurement  Assessments 
Analysis  and  Test  Technologies 
Requirements  Engineering 
Rapid  Prototyping 

Language  Studies/Tool  Development 
Standards  and  Specifications 
Metrics  Data  Clearinghouse 
Software  Quality 
Life  Cycle  Management 


Artificial  Neural  Networks 
Algorithm  Development 
Software  Image  Processing 
Tools  and  Environments 
Cost  and  Reliability  Modeling 
Ri.sk  A.s.se.s.sment 
Program  Management 
Producibiliiy  Measures 
Proces.s/Product  Model  Studies 
Conference  Support 
Software  Technology  Training 
Software  Signal  Processing 


The  following  paragraphs  summarize  the  thirty  three  special  studies  conducted  by  the  DACS 
staff  for  the  defense  software  engineering  community.  Additional  information  may  be  available 
concerning  a  particular  study  by  contacting  the  DACS  Director. 


lO.l  Knowledge  Based  Systems  Evaluation  for  Military  Man-in-Space 

U.S.  Air  Force  Space  Command  (USAF  SPACECOM/XPSS) 

Task  9554F  2010 

Air  Force  Space  Command  wa.'i  reviewing  concepts  for  supporting  war  fighting  in  near-real  time 
with  information  from  space-based  sen.sors.  Such  a  review  included  technology  asses.sments  of 
sensor,  communications,  and  computer  hardware,  and  assessments  of  systems  technologies  for 
fault  tolerant  networking,  smart  distributed  computing,  planning  and  scheduling  aids,  .sensor 
fusion,  and  .sensor  data  reduction.  This  effort  was  referred  to  as  Advanced  Data  Acquisition, 
Processing  and  Transmission  Center  (ADAPT-C).  The  DACS  has  assisted  Air  Force  Space 
Command  by  determining  how  artificial  intelligence  (Al)  techniques  could  contribute  to  the 
ADAPT-C  concept. 

In  the  near  term,  the  best  way  to  demon.strate  AI  contributions  was  through  the  u.se  of  Al  ba.sed 
simulations.  The  AI  simulator  not  only  provided  a  more  u.ser  friendly  simulation  environment. 
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but  it  enabled  the  demonstration  of  AI  methods  in  the  target  system  being  simulated.  For 
example,  AI  based  scheduling  can  be  demonstrated  by  first  simulating  a  specific  satellite 
constellation;  scheduling  tasks  may  then  be  generated  in  the  context  of  this  simulation.  An  AI 
inference  based  scheduler  can  then  be  implemented  to  solve  the  scheduling  problem.  Finally,  the 
AI  scheduler’s  performance  can  be  compared  to  a  more  traditional  numerical  methods  scheduling 
approach. 

"Active  vision"  ideas  can  be  analyzed  with  the  satellite  simulation  tool.  For  example,  if  one 
sensor  spots  an  object  and  requests  confirmation  or  increased  resolution  from  a  second  sensor, 
the  simulator  can  evaluate  the  best  case,  average  case  and  worst  case  availability  of  the  second 
sensor. 

The  simulation  tool  may  also  be  valuable  for  evaluating  the  various  land,  air  and  space  basing 
options  for  an  ADAPT-C  center.  The  relatively  easy  reconfiguration  capabilities  of  an  AI 
simulator  can  more  easily  accommodate  this  analysis  than  would  be  the  case  with  traditional 
simulation  tools. 

10.2  Support  Services  for  the  5th,  6th,  and  7th  Annual  Rome  Laboratory  Knowledge- 
Based  Software  Assistant  (KBSA)  Conference 

U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/C3C) 

Task  9554F  2020 

This  study  supported  and  tracked  the  progress  of  the  development  and  activities  surrounding 
Rome  Laboratory’s  program  for  the  development  of  a  Knowledge-Based  Software  Assistant.  The 
paradigm  is  based  on  formalization  and  machine  capture  of  all  software-related  decisions  and 
subsequently  applying  knowledge-based  reasoning  to  assist  with  decision  making. 

The  DACS  provided  complete  conference  support  for  Rome  Laboratory  from  initial  site  survey 
to  site  selection,  host  service,  and  publication  and  distribution  of  the  resulting  Conference 
Proceedings. 

The  conferences  were  highly  successful  and  attendance  expanded  considerably.  The  conference 
acceptance  within  the  user  community  led  partially  to  a  decision  to  hold  the  conferences  in  more 
accessible  areas. 

10.3  Investigation  of  Automated  Data  Synthesis  Techniques 

U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/OCTS) 

Task  9554F  2030 

This  DACS  special  task  investigated  new  and  novel  approaches  for  software  implementations 
and  related  algorithms  for  the  proce.ssing  of  .signals,  information,  and  knowledge  from  multiple 
and  diver.se  collection  points.  During  the  course  of  this  investigation  systems  with  an  architecture 
of  distributed  data  collection  in  which  the  data  contained  a  wide  variety  of  features/parameters 
arising  from  the  same  phenomena/event  were  considered.  The  algorithm  resulting  from  this 
research  provided  for  real-time  update  of  the  information/data  synthesis  process  and  related  data 
processing  to  a.ssure  optimum  system  performance  in  a  dynamic  environment  of  changing 
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information  acquisition.  The  algorithms  developed  under  this  effort  were  developed  with 
provisions  for  defining  data  flow  constraints  between  the  data  collection  point  and  data  synthesis 
point. 


In  the  development  of  algorithms  for  such  distributed  information  collection  systems,  we 
developed,  with  assistance  from  Dr.  Parmod  Varshney  of  Syracu.se  University,  and  made  u.se  of 
bounds  on  the  following: 

•  Algorithm/system/software/hardware  performance 

•  Algorithm/system/software/hardware  design  methodology 

•  Known  entropy  constraints  and  other  relevant  information  theory 
concepts 

•  Investigation  of  the  application  of  knowledge  based  techniques 

•  Investigation  of  the  application  of  high  level  knowledge  processing 
techniques  such  as  blackboard  sy. stems 

•  Investigation  of  production  systems  which  operate  from  a  rule  ba.se 
for  the  control  of  data  collection 

•  Investigation  of  distributed  detection  of  information. 


10.4  Software  Quality  Automated  Method  Validation:  Setup  &  Support  Task 

U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/C3C) 

Task  9554F  2040 

In  this  DACS  special  task,  the  DACS,  with  support  from  a  subcontractor.  Software  Productivity 
Solutions,  defined  and  established  a  support  function  to  form  the  ba.sis  of  a  larger  effort  titled 
"RADC  Software  Quality  Initiative."  This  program  combined  the  resources  available  from  past 
efforts  performed  by  Rome  Laboratory,  included  efforts  accomplished  or  sponsored  by  Rome 
Laboratory,  and  continued  with  the  direction  undertaken  by  Rome  Laboratory.  The  common 
thread  which  bounds  these  efforts  is  research  in  the  area  of  software  quality. 

The  Laboratory  Support  Function  exists  to  provide  a  mechanism  to  acquire  a  wide  variety  of 
technological  support.  It  further  serves  to  provide  initial  definition  of  a  structure  for  the  lab  and 
its  operation,  as  well  as  a  physical  location  for  the  retention  of  operational  hardware  and  data. 

The  Laboratory  Support  Function  provided  the  management  of  overall  technical  support  needed 
to  oversee  laboratory  functions  and  to  carry  out  specific  subtasks.  This  function  also  included 
the  preparation  and  development  of  materials  and  capabilities  needed  to  support  the  sub  task 
requirements  for  user  training,  installation  and  operation  of  software  quality  tools,  expert 
consultation  on  the  software  quality  tools  and  RSQF  (Rome  Laboratory  Software  Quality 
Framework),  experiment  design,  and  to  analyze  quality  data  and  evaluate  the  RSQF  and  tools. 
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The  preparation  of  sub  task  plans  provided  details  concerning  each  demonstration/application  or 
experiment  that  is  to  be  performed.  The  subtasks  consisted  of  activities  that  included  installation 
and  operation  of  the  tools,  user  training,  collecting,  analyzing  and  interpreting  quality  data,  and 
evaluating  the  technology.  Specific  requirements  for  each  sub  task  was  provided  by  Rome 
Laboratory  as  test  projects  become  available. 

Two  categories  of  subtasks  were  anticipated.  One  required  the  installation  and  operation  of  the 
RSQF  and  tools  at  various  contractor's  plants.  Contractor/support  personnel  were  active 
participants  in  the  demonstrations.  With  the  assistance  of  the  Laboratory  Support  Function 
contractor  (and  subcontractors  as  required),  they  operated  the  tools,  collected  and  analyzed  the 
data,  etc.  Roles,  requirements,  and  duties  were  established  for  all  participants. 

The  second  type  of  sub  task  was  intended  to  minimize  project  interference  and  required  little 
involvement  on  the  part  of  the  software  developer  personnel.  Copies  of  software  products 
generated  by  the  project  were  provided  by  the  developer,  but  the  experiment/demonstration  sub 
task  was  performed  by  personnel  available  through  the  support  function. 

10.5  Software  Engineering  for  Technical  Data  Package  &  Engineering  Change  Proposal 
System 

U.  S.  Army  Research  and  Development  Engineering  Center  (ARDEC)/Battlefield  Automation 
Technical  Division  (BATDD) 

Task  9554F  2050 

The  main  objective  of  this  DACS  special  ta.sk  was  to  provide  software  engineering  expertise  to 
the  Technical  Data  Directorate  at  U.S.  Army  Research  and  Development  Engineering  Center 
(ARDEC),  Dover,  New  Jersey,  in  their  effort  to  develop  a  prototype  system  to  handle  the 
electronic  routing  of  engineering  drawings  and  related  forms.  This  involved  assisting  in  solving 
problems  related  to  the  loading  of  engineering  drawings  to  an  optical  disk  storage  system,  advice 
on  the  design  of  a  configuration  management  system  to  handle  engineering  changes  to 
documents  being  routed,  and  recommend  enhancements  and  performance  improvements  to  the 
system's  user  interface. 

The  front-end  tools  and  languages  currently  being  u.scd  in  the  development  of  the  user  interface 
to  the  Technical  Data  Package  (TDP)/  Engineering  Change  Proposal  (ECP)  system  with  regard 
to  ease  of  use,  maintainability,  response  time  and  portability  were  reviewed.  The  TDP/ECP 
system  was  being  developed  using  an  older  (1970's)  forms  and  menus  package  for  which  the 
government  owns  the  source  code.  This  package  lacked  the  features  of  more  modem  packages 
such  as  pop-up  windows,  sliding  menus,  multi-window  view,  etc.  The  TDP/ECP  system 
software  included  'C  interfaces  to  other  Army  systems  and  u.se  Structure  Query  Language 
(SQL)  for  database  queries. 

The  engineering  drawings  being  scanned  with  the  current  technology  had  several  problems 
which  were  analyzed.  For  example,  older  engineering  drawings  contained  lines  drawn  in  black 
pen,  lead  pencil,  colored  pens  and  pencils'  etc.  and  were  frequently  stained.  The  various 
intensities,  line  thicknesses  and  shades  of  gray  made  the  .scanning  of  the.se  documents  a  difficult 
process;  settings  at  either  end  of  the  .scale  cau.sed  markings  at  the  other  end  to  disappear  or  bleed. 
New  commercially  available  .state-of-the-art  scanning  hardware/.software,  .such  as  high  re.solution 
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flatbed  color  scanners  and  aperture  card  readers,  which  could  improve  the  customer’s  scanning 
rate  of  success  were  examined. 

10.6  Develop  Methodology  and  Evaluation  Criteria  for  Conversion  of  Technical  Data 
Package  and  Engineering  Change  Proposal  System 

U.  S.  Army  Research  and  Development  Engineering  Center  (ARDEC/BATDD) 

Task  9554F206() 

This  DACS  special  task  was  concerned  with  identifying  the  best  commercially  available  front- 
end  development  tool  to  be  used  in  the  Technical  Data  Package  (TDP)  conversion.  The  U.S. 
Army  Research  and  Development  Engineering  Center  (ARDEC)  located  at  Dover,  New  Jersey 
developed  a  Distributed  TDP/Engineering  Change  Proposal  (ECP)  Tracking  and  Reporting 
System  which  satisfied  the  following  main  objectives:  to  improve  response  time  and  quality, 
provide  absolute,  rapid  accountability,  management  oversight  and  notifications  of  pending  TDPs 
and;  to  provide  for  paperless  routing  of  documents  through  the  ECP  process,  to  include 
drawings,  handwritten  and/or  electronic  documents  in  order  to  shorten  the  process. 

The  current  TDP/ECP  system  software  was  developed  using  an  older  (1970’s)  forms  and  menus 
package  that  was  difficult  to  maintain  and  required  a  long  learning  curve  for  programmers.  To 
make  the  system  more  maintainable  and  portable  to  other  hardware,  operating  systems  and 
databases,  the  TDP/ECP  software  was  converted  to  use  a  modern  interactive  and  easy  to 
understand  development  system  that  doesn't  require  a  high  degree  of  traditional  programming 
and  that  is  easily  ported  across  multi-vendor  hardware  and  software  environments. 

Several  leading  commercially  available  front-end  menus  and  forms  products  were  reviewed  for 
functionality  and  compatibility  with  the  existing  hardware  and  databa.se  systems.  From  the  field 
of  available  packages,  three  candidates  were  cho.sen  for  analysis  of  performance  factors  defined 
specifically  for  the  TDP/ECP  application  and  its  u.scr  community. 

User  acceptance  of  an  automated  forms  .system  is  determined  by  factors  such  as  system 
availability,  dependability,  responsiveness,  "friendliness”,  in  addition  to  the  other  system 
performance  factors  such  as  maintainability,  functionality,  portability,  iran.saciion  rates,  network 
connectibility,  database  compatibility,  etc.  An  evaluation  suite  for  the  TDP/ECP  front-end 
software  was  evaluated,  as  well  as  the  development  of  prototypes  using  each  of  the  three 
candidate  front-end  tools  against  this  suite. 

10.7  Man-Machine  Interface  Experiment 

U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/OCTM) 

Task  9554F  2070 

This  DACS  special  task  examined  and  a.s.se.s.sed  the  Rome  Laboratory  Software  Quality 
Framework  '^RSQF)  in  the  course  of  upgrading  a  currently  existing  multichannel  signal 
processing  .s  iware  system.  The  new  version  of  the  signal  processing  .software  interfaced  with 
the  user  by  means  of  the  User  Front-end  Interface  (UFl)  as  a  component  of  the  Multiscnsor 
Algorithm  Experiment  (MAX). 
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Rome  Laboratory/C3C  has  sponsored  research  over  the  last  decade  that  has  resulted  in  a 
collection  of  software  metrics  intended  to  assess  such  management-oriented  quality  factors  as 
reusability.  Controlled  experiments  focusing  on  narrow  aspects  of  the  Software  Quality  Metrics 
framework  were  performed,  creating  a  foundation  for  validation.  Two  software  systems  for 
Rome  Laboratory/OCT  were  developed,  which  provided  a  platform  for  assessing  the  metrics 
under  one  quality  factor,  namely  reusability.  These  software  systems  can  also  be  used  as  a 
mechanism  for  developing  a  model  for  estimating  the  cost  of  applying  the  framework. 

The  existing  multichannel  signal  processing  software,  developed  for  Rome  Laboratory/OCTM, 
synthesizes  and  analyzes  multichannel  signals.  The  signals  consist  of  sums  of  auto-regressive 
processes  and  white  noise,  while  the  analysis  consists  mostly  of  time  domain  methods.  The 
functionality  of  this  system  meets  its  primary  requirement,  exploring  the  false  alarm  rate  and 
probability  of  detection  of  a  new  signal  detection  algorithm.  The  other  system,  the  Multi.sensor 
Algorithm  Experiment  (MAX)  system,  was  developed  for  Rome  Laboratory/OCTS.  A 
sophisticated  man-machine  interface  for  this  system,  the  User  Front-end  Interface  (UFI)  was  al.so 
developed. 

The  purpose  of  the  modification  effort  was  to  create  a  new  version  of  the  multichannel  signal 
processing  software  under  the  UFI.  TTie  new  system  was  developed  by  reusing  as  much  code  as 
possible  from  the  currently  existing  .system.  During  the  modification  effort,  data  was  collected  to 
perform  a  quantitative  assessment  of  the  capabilities  of  the  Rome  Laboratory  Software  Quality 
Metrics  to  measure  reusability. 

10.8  Development  of  Standards  and  Configuration  Management  for  Army  Open  Systems 
Architecture 

U.S.  Army  ARDEC/BATDD 
Task  9554F  2080 

This  DACS  special  task  aided  in  the  review  and  extension  of  Army  MIS  standards  as  they  apply 
to  the  Technical  Data  Package  (TDP)/Engincering  Change  Proposal  (ECP)  operating 
environment.  Another  function  of  this  DACS  special  ta.sk  focused  on  the  development, 
implementation  and  evaluation  of  a  configuration  management  plan  for  the  TDP/ECP  operating 
environment. 

The  new  Army  systems  being  developed  for  the  1990’.s  and  beyond  will  encompass  the  concept 
of  open  systems  architecture.  Where  the  existing  Army  .standards  address  standalone  hosts  with 
resident  applications  and  databases,  the  new  .standards  need  to  addre.s.s  distributed  applications 
and  databases  running  in  a  network  environment  on  heterogeneous  machines.  Such  standards 
will  address  concepts  such  as  the  following:  communication  protocols  for  both  local  area  and 
wide  area  networks  (TCP/IP,  X.25,  SNA,  etc.)  and  network  management;  databa.se  query 
languages  (SQL,  4GL,  etc.);  operating  sy.stem  interfaces  (POSIX,  X/OPEN,  AT&T  System  V 
Release  4,  BSD  4.3,  OSFl,  etc.);  terminal  interfaces  (VTKK),  3270,  PCs  running  emulation 
software,  etc.)  and;  page  de.scription  languages  (POST.SCRIPT,  HPGL,  EPSON,  DIABL(3,  etc.). 

The  new  Army  systems  incorporate  not  only  the  new  hardware  and  software,  but  al.so  make  u.se 
of  the  exi.sting  hardware  and  some  .software  systems.  This  "incorporation"  includes  the  concept 
of  interchangeable  parts  where  the  application  behaves  consistently  across  hardware  and 
database  platforms. 
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The  Technical  Data  Package/Engineering  Change  Proposal  (TDP/ECP)  system  is  moving  to  an 
open  systems  architecture.  The  DACS  special  task  was  designed  to  run  the  application  software 
on  a  variety  of  host  machines  (SUN,  AMDAHL,  UNISYS,  DEC),  perhaps  concurrently,  and  to 
interface  with  several  databases  (SYBASE,  ORACLE,  INFORMIX,  PROGRESS).  Since  this 
system  is  the  first  of  its  kind  being  developed  for  distribution  to  heterogeneous  environments,  a 
configuration  management  plan  was  developed  that  regulates  not  only  the  application  software 
but  also  the  environments  in  which  it  operates. 

10.9  GTD/Scatter  Code  Integration  and  Assessment  Program 

U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/RBCT) 

Task  9554F2{)9() 

A  variety  of  computer  programs  exist  in  many  application  areas.  Such  programs  will  not  remain 
in  use  long  if  they  are  not  integrated  with  new,  more  efficient  algorithms  as  they  are  developed. 
Likewise,  the.se  application  systems  need  to  be  transitioned  to  take  advantage  of  new,  more 
powerful  architectures  as  they  become  available.  Such  enhancements  and  modifications  should 
retain  the  advantages  of  the  original  design,  while  making  use  of  improvemenLs  in  software 
engineering  practices  made  since  the  original  implementations.  This  DACS  special  task 
examined  these  issues  in  the  context  of  software  systems  for  electromagnetic  effects,  especially 
the  General  Electromagnetic  Model  for  the  Analysis  of  Complex  Systems  (GEMACS); 
integrating  new  algorithms  into  these  software  systems  and;  modifying  them  to  run  on  advanced 
parallel  architectures. 

Rome  Laboratory/RBCT  has  sponsored  the  development  of  a  variety  of  software  systems  for 
investigating  electromagnetic  effects  such  as  High-Power  Microwaves  (HPM),  Electromagnetic 
Interference  (EMI),  Electromagnetic  Pulse  (EMP),  lightning,  Electronic  Countermeasures 
(ECM),  sidelobe  susceptibilily/vulnerability  jamming,  and  ultra-wideband  (UWB)  effects.  One 
of  the  most  powerful  and  fiexible  of  these  computer  programs  is  the  General  Electromagnetic 
Model  for  the  Analysis  of  Complex  Systems  (GEMACS).  The  problems  attacked  by  GEMACS 
were  of  particular  concern  among  Atmospheric  Defense  Initiative  (ADI)  antennas  and  platforms. 
To  keep  GEMACS  in  continual  use,  however,  recent  advances  in  engineering  understanding  of 
electromagnetic  effects,  numerical  algorithms,  and  computer  architectures  had  to  be  integrated 
with  GEMACS  to  produce  a  more  powerful  software  system. 

The  results  of  this  study  were  documented  in  a  technical  report  di.scu.ssing  the  problems  rai.sed  by 
such  enhancements  and  integration,  and  recommending  software  engineering  programming 
practices  and  coding  standards  for  dealing  with  ihe.se  problems. 


10.10  Investigation  of  the  Quaiity  of  Signal  Frocessiiig  Software 
U.  S.  Air  Force  Rome  Laboratory  (USAF  RL/OCTS) 

Task  9554F2100 

Rome  Laboratory/C3C  had  developed  a  methodology  for  assessing  the  quality  of  software,  the 
Rome  Laboratory  Software  Quality  Framework  (RSQF),  but  had  not  yet  developed  extensive 
experience  with  it  in  any  application  area.  Rome  Laboratory/OC  developed  software  for  .signal 
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processing  systems  and  algorithms,  thus  providing  a  candidate  application  area  for  validating  the 
RSQF.  Tliis  DACS  special  task  applied  the  RSQF  to  a  signal  processing  system  and  algorithms 
developed  by  Rome  Laboratory/OC.  The  Quality  Evaluation  System  (QUES),  a  software 
measurement  tool  developed  by  Rome  Laboratory/COEE,  was  used  as  an  aid  in  applying  the 
RSQF. 

The  main  objective  was  to  develop  a  few  signal  processing  algorithms  to  provide  a  test  bed  for 
the  RSQF  and  QUES.  The  algorithms,  developed  by  Syracuse  University  Professor  Dr.  Hong 
Wang  under  subcontract  to  the  DACS,  were  developed  to  address  the  problems  of  statistically 
characterizing  a  radar  signal  environment,  developing  optimal  adaptive  space-time  processing 
methods  and  optimal  local  detectors.  These  algorithms  explored  the  implications  of  various 
system  architectures  and  the  environment  in  which  a  signal  processing  system  must  operate. 

10.11  Ada  9X  Implementation  Analysis  Support 

U.S.  Air  Force  Systems  Command  (SC/AFAL) 

Task  9554F2120 

The  Department  of  Defense  mandated  the  use  of  the  Ada  programming  language  on  all  mission 
critical  systems.  Proper  use  of  Ada  requires  the  application  of  certain  important  .software 
engineering  concepts.  Hence.  Ada  is  a  major  vehicle  for  the  transition  of  software  engineering 
techniques  within  the  defen.se  community. 

The  American  National  Standards  Institute  (ANSI)  and  the  Department  of  Defen.se  procedures 
require  that  action  be  taken  periodically  to  reaffirm,  revise,  or  withdraw  the  Ada  language 
standard,  ANSI/M1L-STD-1815A.  The  Ada  Joint  Program  Office  (AJPO)  had  determined  that  a 
revision  was  necessary.  On  25  October  1988,  the  Director  of  the  AJPO  announced  the  initiation 
of  the  revision  process,  referred  to  as  the  Ada  9X  Project. 

The  overall  goal  of  this  DACS  special  ta.sk  was  to  revise  ANSI/MIL-STD- 18 15A  to  refiect 
current  es.sential  requirements  with  minimum  negative  impact  and  maximum  positive  impact  to 
the  Ada  community.  The  Ada  9X  Proce.ss  was  a  revi.sion  and  not  a  redesign  of  the  language  and 
should  be  viewed  as  a  natural  part  of  the  language  maturation  process. 

Requirements  for  the  Ada  9X  revision  proce.ss  were  developed  by  the  Ada  9X  Project 
Requirements  Team  from  New  York  University.  The.sc  requirements  were  be  based  on  revision 
requests  from  the  Ada  community,  workshop/public  meeting  results,  military  .service  \da 
waivers,  Ada  9X  Project  focu.sed  study  results,  and  public  comments. 

The  Mapping/Revision  Team  mapped  the  requirements  developed  by  the  Requirements  Team 
into  recommended  .solutions,  and  revi.sed  AN.S1-M1L-.STD-1815A  based  on  the.se  .solutions. 


10.12  Prototype  Software  Metrics  Analysis  &  Project  Management  Tool  Investigation 
U.  S.  ARMY  Operational  Test  &  Evaluation  Command 
Task  9554F2130 

The  purpose  of  this  DACS  special  task  was  to  develop  a  computer-based  prototype  tool  for 
assessing  and  analyzing  software  metrics  in  support  of  the  U.S.  Army  Operational  Test  & 
Evaluation  Command  (OTEC)  mission.  Tasks  were  performed  by  meml^rs  of  the  DACS  staff 
and  a  DACS  subcontractor.  Dr.  Amrit  Goel  of  Computer  Software  Modeling  Associates.  The 
analyses  are  being  performed  on  Army-wide  metrics  databases  to  be  generated  for  various  Army 
Information  Systems  (AIS)  and  Material  Systems  Computer  Resource  (MSCR)  projects.  This 
analysis  is  important  to  assess  the  progress  of  a  software  project  and  to  predict  via  analytical 
models,  important  quantities  such  as  software  reliability  and  operational  test  readiness.  Analyses 
are  also  be  performed  to  support  the  management  of  future  projects. 

Several  subtasks  were  pursued  during  the  development  and  implementation  of  the 
aforementioned  prototype  tool.  The  first  task  was  the  development  of  a  database  specification 
and  schema  for  an  Army-wide  software  metrics  database  which  allowed  multiple  users  to  .store, 
maintain  and  retrieve  metrics  associated  with  various  projects.  The  second  task  was  the 
incorporation  of  validation  criteria  to  make  the  metrics  database  credible.  For  example,  two 
different  metrics  for  complexity  should  not  provide  contradictory  or  inconsistent  information 
about  the  software  project  to  a  user  for  the  metrics  database.  This  ta.sk  identified  functionality 
needed  to  provide  automated  support  for  metric  validation,  assessment  and  correlation. 

The  third  task  dealt  with  identifying,  early  in  the  life  cycle,  tho.sc  components  of  an  on-going 
project  in  the  software  metrics  database  which  may  need  special  attention.  This  information  may 
be  sought  from  an  analysis  of  the  requirements  and  design  documents  of  the  project.  Previous 
experience  in  dealing  with  similar  components,  documented  in  a  metrics  database,  was  helpful  in 
the  process.  This  task  developed  a  scheme  for  querying  the  metrics  database  to  produce 
integrated  metric  sets  related  to  target  attributes  and/or  goals. 

The  fourth  task  focused  on  the  development  of  an  analytical  framework  for  identifying  problem 
areas  and  tracking  trend  indicators  for  .selected  metrics.  The  fifth  task  identified  salient  features 
of  an  on-going  project  which  may  be  u.sed  to  query  the  metrics  databa.se  for  similar  project 
histories.  When  appropriate  features  are  identified,  much  can  be  learned  from  previous  projects 
to  predict  and  manage  new  ones. 

The  previous  tasks  identified  the  required  functionality  of  a  prototype  software  metrics  analysis 
and  project  management  tool.  In  addition,  they  developed  methods  for  providing  that 
functionality.  The  sixth  task  implemented  those  methods  in  a  running  prototype,  which  may  be 
the  basis  for  developing  a  production  quality  tool  at  a  later  date.  The  .seventh  ta.sk  explored  the 
similarities  and  differences  between  Army  and  Air  Force  metric  collections. 
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10.13  Evaluate  Conversion  of  the  A  RDEC  A  MAS 
U.S.  Army  ARDEC/BATDD 

Task  9554F2140 

The  U.S.  Army  Research  and  Development  Engineering  Center  (ARDEC)  located  at  Dover,  New 
Jersey  currently  is  operating  an  office  automation  system  called  the  Automated  Materiel 
Acquisition  System  (AMAS),  which  provides  ARDEC  personnel  with  an  electronic  means  of 
submitting  and  tracking  requests  for  supplies  and  non  expendable  materiel  (under  $25,(XX).(X)). 
AMAS  was  developed  with  the  objectives  of  reducing  the  administrative  lime  involved  in  the 
requisitioning  process,  reducing  errors  in  the  process,  and  providing  requisitioners  with  an 
automated  tracking  mechanism. 

The  current  AMAS  has  serious  portability  and  maintainability  problems.  Porting  the  system  to 
another  flavor  of  Unix,  or  to  another  hardware  platform,  has  historically  taken  from  six  to  twelve 
months.  The  system  is  tightly  bound  to  proprietary/non  commercial  user  interface  software 
developed  in  the  1970's,  and  has  proven  extremely  difficult  to  debug  and  enhance. 

The  Technical  Data  Package  (TDP)/Engineering  Change  Proposal  (ECP)  system  has  been 
prototyped,  using  a  databa.se  driven  u.ser  interface.  The  conversion  approach,  using  tools 
developed,  resulted  in  substantial  labor  .savings  and  error  reduction. 

This  DACS  special  task  met  the  following  objectives;  generalize  and  extend  the  tools  developed 
under  previous  studies  dealing  with  the  TDP/ECP  system;  evaluate  these  tools  in  the  context  of 
converting  the  AMAS  system  and;  develop  a  functioning  AMAS  prototype  that  is  more 
maintainable  and  portable  than  the  existing  .system. 

10.14  Operation  &  Maintenance  of  the  Robotics  &  AI  Database  (RAID) 

U.S.  Navy  Naval  Oceans  Systems  Center 
Task  9554F2150 

The  RAID  database  was  designed  as  a  information  .support  system  for  program  managers  and  as 
a  technical  knowledge  ba.se  for  robotics  and  AI  re.searchers.  Operation  and  maintenance  of  this 
database  was  taken  over  by  the  DACS  from  Computer  Sciences  Corporation  with  their  transition 
support,  as  a  special  task  through  the  Naval  Ocean  Systems  Center  and  funded  jointly  by  the 
services.  In  the  future  it  may  become  a  CORE  activity. 

It  contains  an  electronically  acce.ssiblc  collection  of  project  abstracts.  The  information  stored  in 
RAID  is  gathered  from  and  verified  by  a  variety  of  sources.  The  topics  included  in  the  daiaba.se 
are  retrievable  by  keywords  in  a  broad  range  of  technical  fields  and  by  system  applications  of 
interest.  PROJECT  INFORMATION  -  includes  a  description  of  research  (both  an  abstract  and  a 
classification),  responsible  DoD  organization,  performing  organization,  names  and  phone 
numbers  of  the  points-of-contact,  and  funding  information.  CONTACTS  INFORMATION  - 
includes  the  addre.ss,  phone  number,  intere.st  areas,  and  electronic  mail  address  of  individuals  and 
organizations  involved  in  robotics  and  AI  re.search  and  development.  Database  u.ser.s  can  .select 
and  combine  information  to  meet  their  needs  by  direct,  on-line  access  or  by  requesting  the  RAID 
staff  to  tailor  report  formats  to  meet  individual  u.ser  requirements. 
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The  information  contained  in  RAID  is  considered  Military  Critical  Technology  (MCT).  There 
are  currently  more  than  2068  projects  and  3143  contacts  records  in  RAID  alliliated  with  840 
valid  organization  records.  RAID  offers  services  and  products  to  provide  the  user  with  an 
increased  information  base  and  increased  on-line  capabilities.  The  lollowing  serviee.s  and 
products  are  currently  provided  at  no  cost  to  users; 

•  On-line  RAID  Access 

•  lAC  Staff  Search  and  Report  Generation 

•  Executive  Summary  Report 

•  Related  Projects  Executive  Summary  Report 

•  Electronic  Communication 

•  Events  Calendar 

•  Conference  Summaries 

•  Mailing  Address  Data  Files 


10.15  Signal  Processing  for  Remotely  Located  Sensors 
U.S.  Air  Force  (USAF  RL/OCTS) 

Task  9554F216() 

This  DACS  special  task,  conducted  with  subcontractor  support  from  Syracuse  University 
Professor,  Dr.  Parmod  Varshney,  investigated  distributed  systems  data  synthesis  and  signal 
management  techniques.  The  main  objective  was  the  derivation  and  development  ol  novel 
software  approaches  for  multi-sensor  signal  processing  and  data  fusion  applications.  Problems  in 
this  area  involve  the  merging  of  diverse  information  from  different  sources  (i.e.  RF,  IR)  into 
coherent  representations  of  the  processed  scene.  These  representations  included  the  detection, 
estimation,  classification  of  the  processed  scene,  and  the  control  of  the  multi-function 
signal/information  processing  systems.  Research  involved  in  this  study  is  geared  towards 
providing  alternative  means  to  using  classical,  Bayesian,  or  Minimax  decision  approaches 
previously  applied  to  this  problem.  The  methods  u.sed  made  u.se  of  heuristic  as  well  as  analytical 
means  for  the  derivation,  development  and  formulation  of  these  algorithms. 

This  project  improved  software  practices  for  multi-sensor  signal  prcKCssing  and  data  fusion  by 
developing  new  methodologies  and  applying  them  to  application  areas  of  great  interest  to  the 
DoD  community. 
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10.16  Independent  Verification  &Validation  Support  for  the  Range  Control  System 
U.S.  Air  Force  (USAF  RUOCDS) 

Task  9554F2170 

The  purpose  of  this  DACS  special  task  was  to  provide  software  engineering  support  to  Rome 
Laboratory(RL)/OCDS.  RL70CDS  was  acting  as  system  integrator  and  software  acquisition 
manager  for  the  Range  Control  System  (RCS),  a  large  computer  system  for  tracking  planes  and 
ships  in  the  Gulf  of  Mexico  during  operation  of  the  Air  Force  practice  range.  The  DACS  and 
subcontractor  Mr.  Dana  Harris  of  Presearch,  Inc.,  supported  ^  l  /OCDS  through  generation  and 
review  of  DOD-STD-2167A  documentation,  Ada  code  review,  ihe  collection  and  analysis  of 
software  quality  metrics  and  management  indicators,  and  Independent  Validation  and 
Verification  (IV&V)  for  document  reviews,  de.sign  reviews,  and  Software  Quality  As.surance 
(SQA). 

RL/OCDS  had  mandated  the  use  of  Ada,  DOD-STD-2167A,  management  indicators  of  problem 
areas,  and  quantitative  measures  of  software  quality  based  on  the  Ro..ie  Laboratory  Software 
Quality  Factor  Framework  (RSQF)  in  the  development  of  RCS  software. 

There  were  several  goals  for  this  study.  The  first  goal  was  to  provide  technical  advice  relevant  to 
RCS  system  integration,  software  acquisition,  and  development  quality  to  RL/fXTDS.  In 
addition,  a  software  quality  measurement  program  was  developed  to  collect  data  needed  for  the 
Air  Force  Management  Indicators  and  for  the  quality  factors  which  may  be  addres.sed  such  as 
Reliability,  Maintainability,  Correctness,  and  Flexibility.  Third,  any  documentation  produced  by 
the  RCS  software  development,  including  DOD-STD-2167A  documents,  was  reviewed.  Next, 
configuration  management  as  defined  by  DOD-STD-2167A  was  monitored  and  the  results 
reported  to  RL/OCDS.  Lastly,  an  independent  a.s.sessment  of  the  Ada  compiler  and  other 
elements  of  the  RCS  software  within  the  development  environment  was  to  be  performed. 

Due  to  loss  of  funding  this  project  was  terminated  prior  to  its  conclusion. 

10.17  Battlefield  Damage  Assessment  Capabilities  and  Evaluation 

U.S.  Army  Mi.ssile  Command  (USA  (AMSMl-SW) 

Ta.sk  9554F218() 

The  purpo.se  of  this  DACS  special  ta.sk  was  to  evaluate  the  applicability  id'  transitioning  a  new 
advanced  software  technology,  known  as  Virtual  Interactive  Prototyping  (VIP),  to  the  DoD  u.ser 
community.  This  was  accomplished  by  developing  a  test  case  VIP  of  the  battlefield  damage 
assessment  (BDA)  mission  for  indirect  fire  .support  smart  weapon  systems  and  evaluating  the 
"applications  development  process"  and  ".software  life-cycle  management"  implications. 

There  existed  several  objectives  in  this  program.  The  primary  objective  was  to  determine  the 
applicability  of  VIP  to  the  weapons  development  community.  Secondary  obiec’iv\..^  included: 
the  definition  of  a  method  for  i.nteracfive  simulation  design  using  knowledge  ha.sed  environments 
(integration  of  object-oriented  programming  and  expert  .systems  into  a  single  .simulation  system), 
the  transition  of  this  technology  to  the  weapons  development  community  and  an  evaluation  of 
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the  spiral  model  of  development  for  rapid  prototyping  as  a  model  of  the  VIP  development 
process. 

The  Army  Materiel  Command  Smart  Weapons  Management  Office  (AMC-SWMO)  served  as 
the  focal  point  for  the  oversig'^l  of  smart  weapons  programs.  These  responsibilities  included 
providing  planning,  technical  evaluations,  recommendations,  coordination  and  in  .some  ca.scs. 
execution  of  smart  weapon  program  activities.  This  included  the  definition  of  the  BDA  technical 
performance  requirements. 

The  BDA  mission  in  today's  battlefield  is  a  complex  operation  which  involves  multiple 
command,  control,  communication  and  intelligence  (C31)  systems.  A  VIP  of  the  decide,  detect 
and  deliver  Smart  Weapons  process  to  include  the  damage  asses.sment  prcKc.ss  will  require  a 
technical  understanding  of  C3C1,  weapon  effectiveness,  target  vulnerability,  kill  assessment, 
sensor  resolution  and  forward  observer  operations  and  capabilities.  The  VIP  was  developed  from 
the  concept  of  operations  definiticn  and  software  requirements  analysis  of  tne  BDA  prt>cess.  The 
concept  of  operations  was  derived  from  a  thorough  requirements  analysis  which  identified  all 
systems  and  subsystems  involved  in  BDA,  the  specific  mission  of  each  system  or  subsystem, 
performance  measurements  for  each  system  or  subsystem  and  system  requirements.  The  concept 
of  operation  formulated  from  the  requirements  analy.sis  Wu  from  the  initial  prototype,  from 
which  a  detailed  prototype  de.sign  will  be  developed. 


10.18  Design  Rapid  Prototype  Approach  to  Software  Simulation  for  ADAPT-C  Ground 
Test 

U.S.  Air  Force  Space  Command  (USAF  AFCS/SSD) 

Task  9554F219() 

The  main  objective  of  this  DACS  special  ta.sk  was  to  support  a  future  concept  evaluation  effort 
for  Air  Force  Space  Command.  The  Air  Force  was  planning  a  large  .software  simulaiion  ground 
test  for  a  future  space  system.  This  evaluation  required  rapid  multiple  reconfiguruiions  of  the 
simulator  to  evaluate  design  options  for  an  extremely  high  bandwidth  sy.stem  of  complex  .satellite 
constellations. 

This  DACS  project  provided  technical  guidance  on  the  use  of  rapid  prototyping  environments  for 
the  planning,  design,  coding,  and  evaluation  pha.ses  of  the  required  software  simulation. 
Emphasis  was  on  object  oriented  programming  approaches  and  on  parallel  computer 
architectures.  Object  oriented  programming  should  facilitate  rapid  reconfiguration  of  the 
simulator.  Parallel  processing  may  be  required  to  simulate  the  extremely  high  bandwidth  of  each 
satellite  while  al.so  simulating  a  large  constellation  of  satellites.  Such  capabilities  are  at  the  .state- 
of-the-art  in  computer  .simulation  technology. 

This  project  was  related  to  an  early  DACS  project  for  Air  Force  Space  Command,  titled 
"Knowledge  Ba.sed  Systems  Evaluation  for  Military  Man  in  Space".  The  first  effort  investigated 
the  possible  applications  of  artificial  intelligcnce(Al)  techniques  to  a  space-ba.sed,  possibly 
manned,  surveillance  system.  The  final  report  recommended  that  A1  methods  be  utilized  in  three 
areas:  knowledge  ba.sed  simulation;  inference  ba.sed  planning,  .scheduling,  and  command  and 
control;  and  knowledge  b.xsed  .sensor  cueing,  perception,  and  fu.sion. 
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This  project  was  most  closely  related  to  the  first  recommendation,  but  it  may  also  impact  tne 
second.  A  software  system  that  uses  rapid  prototyping  for  quick  evaluations  of  alternative 
concepts,  during  the  design  and  development  phases  of  a  complex  space  system,  could  also  be 
used  during  the  operational  phase  to  support  planning,  scheduling,  and  command  and  control 
functions.  In  fact,  the  knowledge  based  simulator  might  become  the  core  of  the  command  and 
control  component  of  the  operational  space  system.  This  project  did  not  build  a  simulation,  but 
did  recommend  software  and  compatible  hardware  facilities  for  a  highly  flexible  knowledge 
based  simulation.  While  not  building  a  complete  simulation  some  simple  components  of  a 
simulation  may  be  implemented  and  were  included  in  the  report  as  examples. 

10.19  Evaluate  the  Maintainability  of  the  TDP  Tracking  System 
U.S.  Army  ARDEC  BATDD 
Task  9554F2200 

The  US  Army  Research  and  Development  Engineering  Center  (ARDEC)  located  at  Dover,  New 
Jersey  is  currently  undertaking  the  development  of  a  Distributed  Technical  Data  Package  (TDP) 
&  Engineering  Change  Proposal  (ECP)  Tracking  &  Reporting  System  for  eventual  use  with  the 
major  commands  of  the  Army  Munitions  and  Chemical  Command  (AMCCOM).  The  following 
are  the  main  objectives  of  the  system: 

•  To  improve  TDP  processing  time  and  data  quality,  promote  immediate  accountability  of 
work,  and  enforce  management  oversight  and  notifications  of  pending  technical  data  packages. 

•  To  upgrade  operational  capabilities  and  provide  an  environment  of  paperless  document 
routing  through  the  ECP  process,  including  drawings,  handwritten  data,  and/or  electronic  forms 
in  order  to  shorten  the  time  to  process  a  TDP. 

Under  a  previous  DACS  Special  Study,  Kaman  Sciences  used  the  JYACC  Application  Manager 
(JAM)  to  produce  the  prototype  TDP  Tracker.  We  found  that  we  could  produce  the  application 
foster  and  more  uniformly  under  the  JAM  environment  than  we  could  with  a  traditional,  third 
generation  language  approach.  The  prototype  has  been  installed  and  tested  by  AMCCOM  at 
several  locations. 

Maintainability  is  a  quality  measure  of  the  ease  with  which  software  can  be  understood, 
corrected,  adapted,  and/or  enhanced.  We  considered  maintainability  during  prototype 
development:  we  noted  areas  of  future  enhancement  and  potential  revision,  discussed  portability 
issues,  and  considered  possible  system  interfaces.  This  task  evaluated  how  quickly  and  easily 
modifications  can  be  made  to  the  prototype’s  software  and  established  mechanisms  to  improve 
maintenance  response  time  where  required.  The  results  of  this  task  were  be  an  input  to  the  task 
of  building  the  full  TDP/ECP  Tracker  System. 
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10.20  Software  Quality  Lab  Support  Task 

U.S.  Air  Force  Rome  Laboratory  (USAF  RL/C3CB) 

Task  9554F50(X) 

This  special  study,  consisting  of  a  variety  of  subs  tasks,  paralleled  the  activities  and  charter  of  the 
DACS  and  continued  to  define  and  establish  support  for  Rome  Laboratory’s  Software  Quality 
Initiative.  This  task  provides  follow-on  support'lo  Rome  Laboratory  from  an  earlier  task  , 
numbered  9554F  2040. 

The  Laboratory  Support  Function  provided  a  mechanism  to  acquire  varied  technical  support  and 
maintain  operational  hardware  and  data.  Also  included  was  the  preparation  of  materials  and 
capabilities  needed  to  support  the  subtask  requirements  for  training,  installation  and  operation  of 
the  following  tools: 

•  Quality  Evaluation  System  (QUES) 

•  Rome  Laboratory  Software  Quality  Framework  (RSQF) 

Also  included  in  this  technical  area  task  was  the  design  of  experiments,  and  quality  data  analysis. 

Substantial  support  was  provided  to  the  DACS  through  subcontractors  Mr.  Ed  Comer  of 
Software  Productivity  Solution,  Inc.,  and  Mr.  Gerald  Murine  of  Metriques,  Inc. 


10.21  Enhancement  of  Multichannel  Signal  Processing  Simulation  System 

U.S.  Air  Force  Rome  Laboratory  (USAF  RL/OCTM) 

Task  9554F  23(X) 

This  special  study  was  related  to  a  previous  task  conducted  by  the  DACS  for  Rome  Laboratory. 
The  study  produced  a  Multichannel  Signal  Processing  Simulation  System  (MSPSS)  that 
supported  the  investigation  of  a  signal  detection  algorithm.  The  study  served  to  demonstrate  the 
integration  and  extension  of  the  User  Front-end  Interface  (UFl)  and  Graphics  U.ser  Interface 
(GUI)  and  fourth  generation  automatic  programming  system,  in  an  application  other  than  that  for 
which  it  was  originally  developed.  Since  the  application  software  for  the  new  MSPSS  was 
largely  developed  from  components  of  a  previous  system,  the  previous  study  also  served  to 
investigate  reuse,  particularly  its  measurement  with  the  Rome  Laboratory  Software  Quality 
Framework  (RSQF). 

The  MSPSS  was  implemented  under  a  modem  GUI  using  windows,  menus,  and  a  pointing 
device.  The  back  end  generates  FORTRAN  programs  out  of  existing  components  specific  to  the 
user’s  description  of  his  analysis.  Thus  the  MSPSS  was  also  an  automatic  programming  system. 

This  task  documented  the  MSPSS  and  otherwise  increases  its  usability.  Since  the  system  used  a 
fairly  self-documenting  GUI,  this  documentation  concentrated  on  preci.sely  describing  the 
algorithms  provided  by  the  system. 
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This  task  was  a  direct  outgrowth  of  a  formal  experiment  which  investigated  software  quality 
metrics. 


10.22  Prototype  Software  Metrics  Analysis,  Project  Management,  and  Risk  Analysis  - 
Phase  2 

U.S.  Army  Operational  Test  &  Evaluation  Command 
Task  9554F2400 

This  special  study  continued  the  work  on  a  prototyf)e  of  an  automated  tool  to  support  the 
assessment  of  operational  effectiveness  and  suitability  of  software  early  in  the  continuous 
evaluation  process.  This  ta.sk  defined  additional  tasks  required  to  expand  the  prototype  tool  to  a 
system  for  Measurement-Ba.sed  Software  Risk  Management  (MBSRM). 

The  following  tasks,  supported  by  DACS  subcontractor.  Dr.  Amrit  Goel  of  Computer  Software 
Modeling  Associates.  Inc.,  were  accomplished  for  this  special  study; 

•  Developed  a  top-level  design  of  the  MBSRM  to  provide  “super  metric.s”  for  risk 
assessment 

•  Provided  enhanced  capability  to  define  the  intcrrclation.ships  of  data  elements  and  to 
perform  statistical  analysis  of  metric  data 

•  Rule-based  schemes  were  examined  to  establish  overall  procedures  for  the  application  of 
measurements  to  drive  software  risk  management 

•  Developed  “information  hiding”  in  the  tool  -  the  concept  that  the  u.scr  will  obtain  added 
output  without  having  to  be  an  expert  in  .software  engineering,  .software  reliability  and 
statistics,  because  of  the  implementation  of  “super  metrics.” 

This  special  study  will  aid  the  UvS  Army  Operational  Test  and  Evaluation  Command  to 
implement  Army  mandated  metrics  collection  and  analysis  programs. 


10,23  Advanced  Distributed  Data  Processing  Techniques  for  Heterogeneous  Network 
Systems 

U.S.  Army  ARDEC/BATDD 
Task  9554F2500 

In  an  effort  to  develop  systems  that  operate  in  an  “open  system.s”  environment,  the  US  Army 
Research,  Development,  and  Engineering  Center’s  Battlefield  Automation  and  Technical  Data 
Directorate  (BATDD)  has  sponsored,  through  the  DACS,  software  Engineering  studies  into  the 
requirements  of  their  systems.  A  product  of  these  studies  has  been  the  development  of 
prototyping  tools  to  be  u.sed  in  generating  advanced  office  automation  systems  which  handle 
electronic  forms  generation  and  routing  algorithms  for  networked  systems  operating  across 
heterogeneous  hardware  and  software  platforms.  It  is  planned  that  the  technology  developed  in 
these  studies  will  be  transitioned  to  solve  other  office  automation  and  management  requirements 
within  the  Army’s  Armament,  Munitions,  and  Chemical  Command  (AMCCOM).  The 
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technology  developed  may  also  be  transferred  to  generic  software  engineering  system 
development  activities  pursued  by  the  DACS. 

The  scope  of  this  special  study  was  to  investigate,  evaluate,  report  and  prototype  advanced 
distributed  data  processing  techniques,  tools,  and  methods  for  networked  systems  with  the 
objective  of  improving  the  operational  capabilities  of  the  BATDD  computing  environment.  It 
included  training  personnel  in  software  measurement  and  maintenance  techniques  to  insure  high 
level  system  performance. 

Specific  tasks  in  this  special  study  were: 

•  Assess  current  state-of-the-art  of  the  system 

•  Investigate,  design  and  develop  new  system  tools  and  toolbox 

•  Transition  technology  by  demonstrating  the  tools  and  prototypes,  classroom 
presentations,  and  documentation  development 

10.24  Image  Processing  Technology  for  Heterogeneous  Networks  Systems 
U.S.  Army  ARDEC/BATDD 
Task  9554F26(X) 

The  U.S.  Army  Research,  Development  and  Engineering  Center  (ARDEC)  at  Picatinny  Arsenal 
in  Dover,  N.J.,  is  mission  responsible  for  the  repositories  which  house  major  weapon  system 
Technical  Data  Packages,  both  hardware  and  software.  Considerable  documentation  contained 
within  a  technical  data  package  consists  of  graphical  data,  the  majority  of  it  in  the  form  of 
engineering  drawings.  The  management  of  these  drawings  is  currently  being  u-ansitioned  from  a 
microfilm-based  system  to  one  based  on  optical  storage  technology. 

Recently  ARDEC  has  been  involved  in  programs  aimed  at  developing  advanced  office 
automation  systems  within  the  context  of  open  systems  architecture  that  handle  electronic  forms 
generation  and  routing  for  networked  systems.  Although  these  systems  currently  handle  only 
ASCII  data,  the  goal  of  ARDEC  is  to  include  all  graphic  data  associated  with  a  technical  data 
package  in  its  routing  for  users  in  approving/concurring  organizations  to  access,  view  and  update 
data. 

This  effort  analyzed  designs  and  prototyped  image  storage,  retrieval,  and  processing  capabilities 
for  wide  area  networks  to  include  the  following: 

•  Analyzed  the  performance  of  network  software  used  to  transfer  large  volumes  of  image 
data  and  identified  deficient  network  software 

•  Applied  software  engineering  methods  to  reconfigure  the  deficiencies 

•  Identified,  recommended  and  prototyped  software  engineering  tools  and  methods 
designed  for  the  visually  oriented  user  interfaces 
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•  Identified  and  recommended  appropriate  software  for  data  storage  based  on  the  latest 
magnetic  and  optical  technologies. 

•  Assessed  the  maintainability  of  current  ARDEC  software  systems  involved  in  the  image 
storage  and  retrieval  process 


10.25  Software  Process  and  Software  Metrics  Program 

U.S.  Army  Operational  Test  and  Evaluation  Center 
Task  9554F  2650 

One  of  the  DACS'  main  functions  is  to  transition  software  technology,  such  as  software  metrics, 
throughout  defense  organizations.  The  United  States  Army  Operational  Test  and  Evaluation 
Command  (OPTEC)  has,  through  the  Software  Test  and  Evaluation  Panel  (STEP),  developed  a 
set  of  four  products  to  improve  Operational  Test  and  Evaluation  within  the  Army.  These 
products  consist  of  a  unified  process,  a  guide  to  OT«&E  regulations,  the  User  Functional 
Description,  and  a  set  of  metrics. 

OPTEC  is  now  beginning  the  implementation  of  the  STEP  recommendations  throughout  the 
Army.  This  plan  defined  a  program  for  providing  technical  support  for  the  implementation  of  the 
STEP  recommendations,  particularly  the  STEP  metrics.  This  task  provided  technical  support  to 
OPTEC's  program  for  implementing  the  STEP  initiative,  concentrating  on  the  software  metrics 
developed  by  STEP.  The  DACS  was  joined  in  this  effort  by  subcontractor.  Dr.  Amrit  Goel  of 
Computer  Software  Modeling  Associates,  Inc. 


Technical  support  consisted  of:. 

•  Supporting  a  conference  to  explain  the  STEP  program  to  Army  personnel 

•  Developing  procedures  and  a  methodologies  for  implementing  the  STEP  metrics 

•  Developing  an  instruction  program  for  the  STEP  metrics  and  for  a  prototype  tool  for 
storing  and  analyzing  metric  data 

•  Conducting  seminars  on  the  software  metrics  and  prototype  tool 

•  Documenting  and  analyzing  comments  and  observations  on  the  STEP  metrics  program 
obtained  throughout  this  work. 

10.26  IRSS/PAAS/ARCREF  Code  Conversion 

U.S.  Air  Force  Rome  Laboratory  (USAF  RL/OC) 

Task  9554F  2660 

Rome  Laboratory/OC  has  sponsored  the  development  of  the  Interactive  Radar  Simulation 
System  (IRSS),  the  Parametric  Antenna  Analysis  System  (PAAS),  and  the  ARC  Reflector  Code 
(ARCREF).  TTiese  systems  provided  comprehensive  simulation  capabilities  for  antennas,  radar 
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and  systems.  A  variety  of  analyses  were  made  possible  with  these  simulaiit)n  eapabiliiies.  Fur 
example,  trade-off  studies  can  be  done  comparing  different  antenna  or  processor  designs. 

IRSS/PAAS/ARCREF  currently  runs  on  a  VAX/VMS-based  minicomputer.  fXT  has  begun 
adopting  Sun  workstations  under  various  Graphical  User  Interfaces  (GUIs)  such  as  SunView  and 
OpenWindows.  This  special  study  facilitated  the  transfer  of  this  recent  computer  technology 
into  OC  by  converting  IRSS/PAAS/ARCREF  to  run  on  a  SPARC-based  (e  g.  SPARCstatiun, 
Sun  4,  etc.)  architecture  under  OpenWindows  and  UNIX  (SunOS). 

Kaman  Sciences  transported  parts  of  the  IRSS  and  PAAS  system  to  a  Sun/UNIX  environment 
under  another  program.  This  DACS  effort  began  with  those  Sun/UNIX  components  as  a 
baseline.  Software  engineering  resource  data  was  collected  for  this  DACS  special  task.  This 
data  was  used  within  an  experiment  for  Kaman  Sciences  participation  in  the  Rome  Laboratory 
Software  Quality  Technology  Transfer  Consortium. 

10.27  Battiefleld  Damage  Assessment  -Virtual  Interface  Prototype  Data  Collection 

U.S.  Army  Smart  Weapons  Management  Office 
Task  9554F27(X) 

Kaman  Sciences  Corporation  worked  to  develop  a  new  .software  engineering  technology  to 
evaluate  complex  systems  using  knowledge-ba.sed  computing  environments,  system  engineering 
analysis,  and  virtual  interactive  prototyping  techniques.  The  project  began  under  IR  &  D 
funding  and  continued  under  a  previou.sly  described  special  task  through  the  DACS.  The 
sponsors,  the  US  Army  Smart  Weapons  Management  Office  (SWMO)  was  specifically  interested 
in  the  applicability  of  this  technology  by  developing  test  case  prototypes  of  a  battlefield  damage 
assessment  for  indirect  fire  smart  weapons.  To  conduct  a  thorough  and  reali.stic  evaluation  of  the 
technology,  test  data  was  required. 

The  objective  of  this  task  was  as  follows: 

•  Determine  lest  data  requirements 

•  Determine  data  collection  techniques 

•  Collect  actual  target  and  damage  signatures  to  evaluate  the  BDA/VIP  in  a  test-bed 
environment. 

This  task  provided  the  DACS  a  rare  opportunity  to  use  software  engineering  tools,  techniques 
and  methods  in  an  operational  environment  critical  to  US  war  fighting  capabilities. 

10.28  Risk  Assessment  Prototype 
U.S.  Army  OPTEC 

Task  9554F2800 

This  effort  was  developed  to  prototype  a  .system  which  would  provide  .software  project  managers 
and  developers  with  knowledge-based  support  to  manage  software  ri.sks  using  the  metrics 
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developed  in  the  Army's  STEP  Program.  The  main  objective  of  this  effort  was  to  develop  a 
prototype  knowledge-ba.sed  system  which  will  guide  and  help  managers  understand  the  status  of 
software  projects  and  make  metrics  based  decisions.  It  was  designed  to  help  identify  high  risk 
items,  find  causes  for  the  high  risk  status,  and  suggest  resolution  techniques.  This  effort  was 
undertaken  by  the  DACS  and  Dr.  C.V.  Ramamoorthy  of  the  University  of  California  at  Berkeley. 

A  prototype  Risk  Assessment  tool  was  developed  using  artificial  intelligence  techniques  to 
include  the  use  of 

•  Truth  Maintenance  Systems 

•  Expert  System  Shells 

Documentation  was  also  developed  for  the  prototype  and  the  commented  code.  It  is  anticipated 
that  after  follow-on  expansion  of  this  effort,  this  prototype  may  be  incorporated  into  the  U.S. 
Army’s  STEP  Metric  tool  al.so  prototyped  by  the  DACS. 

10.29  Rome  Laboratory  Software  Quality  Consortium  Support 

U.S.  Air  Force  Rome  Laboratory 
Task  9554F  2850 

This  special  study  established  the  support  parameters  the  DACS  would  provide  to  the  members 
of  the  Rome  Laboratory  Software  Quality  Consortium. 

The  DACS  performed  the  following  Con.sortium  support  services  as  a  part  of  this  task: 

•  Developed  a  Pilot  Application  Plan  (PAP) 

•  Supported  and  assisted  Con.sortium  members  in  the  design  of  the  pilot  application  plans 
for  applying  the  Rome  Laboratory  Software  Quality  Framework  (RSQF). 

•  Provided  Quality  Evaluation  System  (QUES)  Support.  Maintained  the  QUES  and 
assisted  Consortium  members  in  its  u.se 

•  Supported  Consortium  members  in  their  u.se  of  the  RSQF  and  a.s.sociatcd  data  collection 

•  Assisted  in  the  preparation  of  the  Con.sortium  Newsletter,  planning  and  promotional 
activities,  the  technical  interchange  meeting.s,  and  site  visits. 

•  Assisted  in  data  gathering  and  analysis  of  data.  Conducted  u.ser-surveys,  analyzed 
lessons  learned,  assisted  in  RSQF  Framework  validation,  and  data  repository, 

.sanitization,  interpretation,  and  statLstical  analy.sis. 
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10.30  Distributed  Database/Parallel  Processing  Technology 
U.S.  Army  ARDEC/BATDD 

Task  9554F  2870 

To  increase  the  reliability  and  performance  of  its  systems,  the  US  Army’s  Research, 
Development  and  Engineering  Center  (ARDEC),  is  currently  evaluating  the  concept  of 
transitioning  its  database  systems  to  operate  in  a  distributed  database  environment.  In  an  effort 
to  provide  faster,  more  reliable  performance,  the  Battlefield  Automation  and  Technical  Data 
Division  (BATDD)  is  planning  to  upgrade  the  Sun  dataservers  on  ARDEC's  wide  area  network 
to  a  multiprocessing  model  running  Sun’s  new  Solaris  multiprocessing  operating  system. 

BATDD  required  assistance  in  transitioning  its  mission  critical  software  systems  to  embrace 
these  two  new  technological  advancements.  This  task  supported  BATDD's  efforts  in  the 
transition  of  its  mission  critical  software  systems  to  the  emerging  software  technologies  of 
distributed  database  and  parallel-processing  with  emphasis  on  the  technologies  of  distributed 
database  design,  its  use  in  the  storage  and  retrieval  of  image  data  in  a  distributed  database 
system,  and  the  effects  of  parallel-processing  within  a  distributed  databa.se  environment. 

Specific  objectives  of  this  task  were  as  follows: 

•  Developed  strategies  and  methods  for  tran.sitioning  current  software  systems  to  a 
distributed  database/parallel  processing  environment;  analyzed  and  critiqued  the 
effectiveness  of  these  strategies  and  methods  through  the  u.se  of  prototypes 

•  Researched  new  or  modified  techniques  for  improving  database  performance  in  a 
distributed  database/parallel-processing  environment,  translating  the.se  techniques  into 
specifications  for  automated  tools  and  developing  prototypes  of  these  tools 

•  Investigated,  prototyped  and  evaluated  methods  for  using  distributed  database  technology 
on  parallel-processing  dataservers  for  the  storage  and  retrieval  of  image  data 

•  Produced  a  technical  report  analyzing  the  level-of-effort,  difficulties  encountered  and 
solutions  found  in  transitioning  software  systems  from  an  environment  of  single 
processor  machines  to  one  of  distributed  processing  on  parallel  machines 

10.31  Certification  of  Reusable  Software  Components 

U.S.  Air  Force  Rome  Laboratory  (USAF  RL/C3CB) 

Task  9554F  2890 

The  purpose  of  this  special  study  was  to  develop  a  certification  methodology  for  designating 
various  levels  of  confidence  in  the  quality  of  reusable  software  components.  Based  on  existing 
Rome  Laboratory  software  quality  as.sessment  and  test  &  verification  technology,  planned 
improvements  in  the  technology  as  well  as  the  Army’s  existing  Reusable  Ada  Products  for 
Information  Systems  Development  (RAPID)  certification  criteria,  a  rigorous  multi-level 
certification  methodology  was  developed. 
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Levels  of  certification  achieved  were  based  on  the  results  of  applying  each  analysis 
technique/tool  implementing  these  technologies.  The  task  provided  a  vehicle  for  software 
developers/reusers  to  make  intelligent  choices  regarding  the  selection  of  one  component  over 
another,  or  selecting  reuse  over  original  development.  It  al.so  aided  developers  in  understanding 
how  to  develop  reusable  software  and  attain  the  proper  certification  level  for  the  components 
intended  use.  The  certification  and  framework  were  incorporated  into  the  DoD  Software 
Warehouse. 

State-of-the-art  techniques  and  tools  which  can  be  immediately  applied  as  a  part  of  the 
certification  process  were  be  identified. 

The  certification  framework  was  developed  and  the  possible  levels  of  confidence  defined  along 
with  specification  of  the  criteria  for  each  level. 

What  to  store  and  how  to  store  it  was  defined  for  the  content  and  formal  of  the  information 
developed  resulting  from  the  application  of  the  tools/iechniques  developed. 

Joining  the  DACS  in  completing  this  technical  area  task  was  subcontractor  Research  Triangle 
Institute. 

10.32  Software  Metrics  Tool  Support 

U.S.  Army  OPTEC 
Task  9554F29()0 

This  project  defined  the  tasks  required  in-order-to  provide  software  enginei;ring  project 
metrics  and  management  support  for  the  US  Army’s  Operational  Test  and  Evaluation  Center 
(OPTEC).  The  plan  capitalized  on  the  accomplishments  of  several  earlier  software 
engineering  support  tasks  performed  by  the  DACS  for  OPTEC  to  include  the  prototype 
development  of  an  automated  tool  which  supports  the  assessment  of  operational  effectiveness 
and  suitability  of  software  early  in  the  Continuous  Evaluation  (CE)  process.  Once  again,  Dr. 
Amrit  Goel  of  Computer  Software  Modeling  Associates  supported  the  DACS  in  completing 
this  task.  Among  other  things,  this  ta.sk  utilized  and  updated  the  prototype  metrics  analysis 
database  tool.  This  task  also  capitalized  on  the  activities  pursued  under  the  DACS  task  for 
the  Software  Process  and  Software  Metrics  Program. 

This  task  provided  technical  and  configuration  management  support  to  the  US  Army’s 
Operational  Test  and  Evaluation  Center  (OPTEC)  program  for  implementing  the  Software 
Test  and  Evaluation  Panel  (STEP)  metrics  initiative  by  concentrating  on  the  software  metrics 
developed  by  STEP.  DACS  support  consi.sted  of  the  following  activities; 

•  Prepared  a  STEP  Metrics  Clearinghou.se  Operational  Concept  Document. 

•  Evaluated  .selected  requirements  traceability  tools  for  u.se  in  the  STEP  Metrics 
Program. 

•  Implemented  STEP  Data  Item  De.scripiions  (DIDs)  in  the  prototype  databa.se. 

•  Customized  a  user  interface  for  the  STEP  metrics 
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•  Conducted  demonstrations  and  present  training  workshops. 

Modifications  were  made  to  the  prototype  STEP  Metrics  Database  Tool.  Modifications  involved 
a  Commercial-Off-The-Shelf  (COTS)  database  tool  operating  in  a  UNIX  and  X-W1ND()W 
environment. 

10.33  Risk  Management  Using  Knowledge*Based  Techniques 

U.S.  Army  OPTEC 
Task  9554F  2920 

This  project  provided  continued  software  engineering  support  for  phase  2  of  an  effort  to 
provide  metrics-guided  risk  management  using  knowledge-based  techniques  for  the  U.S. 
Army’s  Operational  Test  and  Evaluation  Command  (OPTEC).  Also  identified  were  the 
tasks  required  to  host  the  OPTEC  metrics  tool  on  a  Kaman  Sciences  Sun  computer  for 
evaluation  and  demonstration  purposes. 

The  main  objectives  of  this  special  task  were  as  follows; 

•  Enhance,  install  and  prove  the  concept  of  the  OPTEC  Metrics  Databa.se  Tool  on  a 
Kaman  Sciences  Corporation  Sun  Computer  System.  OPTEC  supplied  actual 
metrics  data  for  test  and  evaluation  in  order  to  demonstrate  the  tool’s  capability  for 
processing  defined  metrics  sets.  Quality  and  requirements  metrics  were  targeted 
for  initial  development,  followed  by  management  oriented  metrics. 

•  Developed  prototype  knowledge-based  systems  which  guide  and  help  human 
managers  understand  the  status  of  software  projects  and  make  decisions  during  the 
metrics  guided  risk  management  of  software  projects.  They  identified  high  risk 
items,  found  specific  causes  for  them,  and  suggested  resolution  techniques.  With 
the  help  of  knowledge-based  systems  human  managers  can  identify  high  risk  items 
and  control  them  before  they  can  cau.se  serious  problems  in  the  project. 

Technical  support  consisted  of  the  following  activities: 

•  Installed  the  OPTEC  Database  Tool  on  a  KSC  Computer. 

•  Loaded  OPTEC  supplied  data  on  the  restructured  databa.se  -  .set  up  for  STEP  DIDs. 

•  Prepared  database  sub-tools  for  metric  algorithm  analysis. 

•  Created  new  menus  for  STEP  data  proce.ssing. 

•  Tested  and  evaluated  database  performance  using  actual  STEP  metrics. 

•  Completed  tool  documentation  and  prepared  a  user  manual. 

•  Surveyed  and  studied  literature  in  theoretical  and  practical  aspects  of  risk 
management  and  metrics-ba.sed  .software  project  management. 


70 


•  Studied  applicability  ot  metrics  in  the  Army  report  for  risk  management 

•  Developed  a  prototype  of  knowledge-based  tool  for  managing  some  major  risk 
factors,  experimented  with  them  and  evaluated  them. 

•  Demonstrated  the  prototype  and  provided  documentation  for  the  prototype. 

This  technical  area  task  was  performed  by  members  of  the  DACS  staff  and  subcitntractors  Dr. 
Amrit  Goel  of  Computer  Software  Modeling  Associates,  Inc.  and  Dr.  C.V.  Ramamoorthy  of  the 
University  of  California,  Berkeley. 
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11.0  LESSONS  LEARNED  AND  RECOMMENDATIONS 


During  this  contract  period,  the  initial  period  of  operation  of  the  Data  ik  Analysis  Center  lor 
Software  (DACS)  by  Kaman  Sciences  Corporation,  significant  additions  to  the  DACS  database 
holdings  were  made.  The  software  engineering  bibliographic  database  (SEED)  was  expanded 
substantially  from  the  previous  twelve  years  of  DACS  operation.  The  number  of  citations  and 
abstracts  increased  in  this  three  year  period  by  almost  50  per  cent.  Similarly,  the  holdings  for  our 
software  life  cycle  database  (SLED),  the  software  engineering  research  projects  databa.se  (SERF) 
and  the  software  engineering  tools  information  database  (SETl)  also  increa.sed. 

New  sources  of  information  were  found  and  new  methods  of  accessing  iho.se  .sources  were 
developed  and  acquired.  However,  we  found  that  to  make  our  resources  available  and  truly 
usable  by  the  DACS  user  community,  we  must  change  the  way  we  provide  that  information  to 
others.  Consequently,  we  recommend  that  the  following  areas  and  opportunities  be  considered 
for  future  adoption; 

•  Deliver  "searches"  in  hardcopy  and  .softcopy 

•  Dedicate  sub.staniially  greater  amounts  of  funds  to  obtaining  source  materials 

•  Integrate  the  output  capability  of  the  databa.ses  to  deliver  a  product  derived  from 
each  one  of  the  databases  where  appropriate 

•  Explore  opportunities  for  u.se  of  the  National  Technical  Information  System  as  a 
distributor  of  DACS  products 

•  Offer  the  opportunity  to  copyrighted  databases  for  the  DACS  to  act  as  a 
distribution  point  for  the  data 

•  Increase  the  marketing  activities  a,ssociated  with  the  DACS  name  and  reputation 

•  Continue  to  vigorously  pursue  the  acquisition  of  software  engineering  data  from 
non-DoD  sources  such  as  NASA,  the  AIAA  and  corporate  entities. 

We  must  also  work  to  increa,se  the  reputation  and  recognition  of  the  DACS  in  all  areas  of  the 
DoD  and  other  governmental  activities.  Participation  at  high  level  activities  such  as  the  DoD 
Software  Strategy  Planning  Forum,  and  with  the  Navy  in  the  Next  Generation  Computer 
Resources  Program  have  paid  dividends  to  the  DACS.  Increa.sed  participation  with  major 
technology  organizations  and  as.sociations  .such  as  the  IEEE  and  ACM  will  continue  that  trend. 
Submission  of  papers  and  pre.sen rations  to  journals  and  conferences  will  help  as  well. 

Finally,  making  better  use  of  our  subcontractors  breadth  of  technical  experience  will  add  to 
DACS  capabilities  and  reputation  as  well  as  to  our  recognition  in  non-traditional  markets. 
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MISSION 


OF 

ROME  LABORATORY 


Rome  Laboratory  plans  and  executes  an  interdisciplinary 
program  in  research,  development,  test,  and  technology 
transition  in  support  of  Air  Force  Command,  Control, 
Communications  and  Intelligence  (C3I)  activities  for  all 
Air  Force  platforms.  It  also  executes  selected 
acquisition  programs  in  several  areas  of  expertise. 
Technical  and  engineering  support  within  areas  of 
competence  is  provided  to  ESC  Program  Cffices  (POs)  and 
other  ESC  elements  to  perform  effective  acquisition  of 
C3I  systems.  In  addition,  Rome  Laboratory's  technology 
supports  other  AFMC  Product  Divisions,  the  Air  Force  user 
community,  and  other  DOO  and  non-DOD  agencies.  Rome 
Laboratory  maintains  technical  competence  and  research 
programs  in  areas  including,  but  not  limited  to, 
communications,  command  and  control,  battle  management, 
intelligence  information  processing,  computational 
sciences  and  software  producibility,  wide  area 
surveillance/sensors,  signal  processing,  solid  state 
sciences,  photonics,  electromagnetic  technology, 
superconductivity,  and  electronic 
reliability/maintainability  and  testability. 


